Skip to content

3.x: Scylla IT matrix runs the same version in all three lanes #1024

Description

@nikagra

The scylla-integration-tests job in .github/workflows/tests@v1.yml has a
scylla-version: [LTS-LATEST, LTS-PRIOR, LATEST] matrix, but the matrix value never reaches
make resolve-scylla-version. The Get scylla version step passes no SCYLLA_VERSION, so the
Makefile default (SCYLLA_VERSION ?= LATEST, Makefile:5) applies in all three jobs. The matrix
value is used only in artifact, check and log names, so the three lanes look distinct while
running the same server.

Evidence — the interpolated step names of the three Scylla ITs jobs in run
33172055360:

Scylla ITs (LATEST, 8)      ->  Run integration tests on Scylla (2026.2.5)
Scylla ITs (LTS-LATEST, 8)  ->  Run integration tests on Scylla (2026.2.5)
Scylla ITs (LTS-PRIOR, 8)   ->  Run integration tests on Scylla (2026.2.5)

So LTS-LATEST and LTS-PRIOR have never tested what they claim: the driver is covered against
one Scylla version, three times, at three times the runner cost.

The Cassandra job does pass CASSANDRA_VERSION: ${{ matrix.cassandra-version }} to its
Get cassandra version step, so that lane resolves the matrix value correctly.

The fix is one env: entry on the Get scylla version step. Worth doing on its own rather than
inside another change: it will make two lanes exercise older Scylla releases for the first time,
so expect it to surface real test failures, and to add two CCM images to the download cache.

Found while reviewing #1018, which sidesteps the symptom by naming its coverage lanes from the
matrix value rather than from SCYLLA_VERSION.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions