Files
Alex Goodman 7c68081d9c chore: replace the integration tests with the quality gate (#3852)
* chore: replace the integration tests with the quality gate

- the Go integration tests in `test/integration` are gone, and the quality gate moves from `test/quality` into its place
- tasks are now `make integration`, `integration:score`, `integration:update-db` and `integration:clean`, matching vunnel's `integration:` tasks
- the race detector scan that used to ride along with the integration tests is now its own `make smoke-test` task and "Smoke tests" CI job
- labels live in `test/integration/additional-claims`, and the gate reads vunnel's from `tests/integration/additional-claims` (anchore/vunnel#1418)

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* test: check that each SBOM format matches the same as an image scan

- `make integration:sbom-formats` scans a few of the quality gate's images with grype from this checkout, then scans an SBOM of each image in syft-json, spdx-json and spdx-tag-value, and fails when the matches differ
- it uses the DB pinned in `test/integration/gates/pr.yaml`, so the comparison holds still between runs
- a format quirk that changes matches for a known reason is listed against that image and format, and a failure logs how the packages behind the differing matches differ
- runs as the new "Integration tests (SBOM formats)" CI job, in parallel with the gate

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: split the integration tests into quality and api suites

- `make integration` runs both suites: `integration:quality` (the yardstick gate) and `integration:api` (the Go tests against grype's library)
- `integration:score` is now `integration:quality:score`
- CI runs the suites as a matrix in parallel, and one "Integration tests" check requires both

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

* chore: log yardstick progress in CI

yardstick only shows progress on a terminal, so the CI job printed nothing until the verdict. In CI (`CI=true`) it now runs with `-v`.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>

---------

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
2026-10-27 17:38:24 -04:00
..