Skip to content

Fix matrix CCM bootstrap and preserve installation errors - #144

Merged
dkropachev merged 2 commits into
scylladb:masterfrom
dkropachev:fix-matrix-ccm-setup-143
Oct 16, 2026
Merged

dkropachev merged 2 commits into
scylladb:masterfrom
dkropachev:fix-matrix-ccm-setup-143

Conversation

@dkropachev

@dkropachev dkropachev commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Issue

In #143, Jenkins build #30 fails during Apache 3.29.3/protocol v4 setup because GitHub returns HTTP 504 for the scylla-ccm archive. The runner swallows the installation exception and subsequently reports a missing JUnit XML file. The earlier missing cluster.conf warning is handled normally; Apache 3.30.1rc1 completes both protocols with no unexpected matrix failures.

Fix

Forward CCM_DIR into the container and use the already-mounted checkout when bootstrapping each driver environment. Reject an invalid configured directory and retain the archive fallback for direct runs without CCM_DIR.

Propagate dependency-installation failures to the existing failure-reporting path so Argus metadata retains the original error and pytest is not started after failed setup.

The CCM source change applies to environment bootstrap. The Scylla driver’s later dependency synchronization keeps its declared CCM Git source and remains outside this PR.

Investigation findings.

Fixes #143.

Decisions

  • Choice: Limit CCM_DIR changes in this PR to environment bootstrap and relative-path handling. Leave the Scylla driver’s later dependency synchronization unchanged, including its CCM Git source, to preserve existing resolver behavior. This exclusion applies throughout this PR and resumed iterations; no follow-up is scheduled.

CI already mounts a scylla-ccm checkout, but environment bootstrap downloads
master.zip again. A GitHub 504 aborts setup before pytest can run.

Pass CCM_DIR into the container and install that checkout when configured.
Keep the archive fallback for direct runs without CCM_DIR. Re-raise install
errors so failure metadata preserves the original cause instead of reporting
a missing JUnit file.

Fixes scylladb#143
@qodo-scylladb

qodo-scylladb Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🔗 Cross-repo conflicts (1)



🟠 Medium

1. Scylla runs still fetch CCM 🔗 Cross-repo conflict
Description
Run._create_venv() installs CCM_DIR locally, but the subsequent uv sync --group dev resolves
CCM from the Git dependency declared by scylladb/python-driver. On a Scylla-driver run with an
uncached CCM dependency, GitHub can still block setup despite the mounted checkout.
Code

run.py[R155-157]

+            ccm_source = str(ccm_dir)
        basic_packages = ("pytest==7.4.4",
-                          "https://github.com/scylladb/scylla-ccm/archive/master.zip",
+                          ccm_source,
Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Scylla-driver dependency sync still resolves CCM from Git after the matrix installs the mounted checkout.

## Fix Focus Areas
- run.py[151-157]
- run.py[201-207]
- /cross_repos/python-driver/pyproject.toml[39-59]

## Recommended Fix
Coordinate the matrix setup with the driver's development dependency configuration so `uv sync` uses the mounted CCM checkout when `CCM_DIR` is set. Verify setup succeeds with an empty dependency cache when the CCM Git remote is unavailable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗



Resolved findings
1. Relative checkout paths fail direct runs ✓ Resolved
Description
_create_venv() resolves CCM_DIR after Run.run() changes the process working directory to the
driver checkout. When a direct run supplies a valid path relative to its original working directory,
setup instead looks for that path under the driver checkout and can fail before installing
dependencies.
Code

run.py[R152-154]

+            ccm_dir = Path(self.environment["CCM_DIR"]).resolve()
+            if not ccm_dir.is_dir():
+                raise FileNotFoundError(f"CCM_DIR does not point to a directory: {ccm_dir}")
Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A relative `CCM_DIR` is resolved after `Run.run()` changes the working directory, so a valid checkout path supplied to a direct run can point somewhere else during setup.

## Fix Focus Areas
- run.py[150-155]
- run.py[251-259]

## Recommended Fix
Resolve a configured `CCM_DIR` against the original process working directory before `Run.run()` calls `os.chdir()`. Use that saved absolute path when creating the virtual environment, retaining validation and the archive fallback.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Tip of the day
💡 Did you know, you can pick a display mode instead of tweaking every Display option by hand

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗


Powered by Qodo

Run.run() changes the working directory to the driver checkout before
bootstrap, so resolving a relative CCM_DIR there loses its original base.
Save the invocation directory once and pass it to every matrix Run so
later version and protocol combinations use the same checkout.

Exercise both driver types through main with real directory changes and
retain the archive fallback for unset or empty CCM_DIR.
@dkropachev

Copy link
Copy Markdown
Collaborator Author

In reply to

Scylla-driver dependency sync still resolves CCM from Git after the matrix installs the mounted checkout.

Correct. For Scylla driver versions that declare CCM as a Git dependency in the dev group, the later uv sync still resolves that source and can fail when it is uncached and the remote is unavailable. This behavior predates this PR. The agreed scope, recorded in the PR’s Decisions section, is to use CCM_DIR for environment bootstrap and fix relative-path handling while preserving the existing later dependency synchronization. That Scylla limitation remains; no follow-up is scheduled.

@dkropachev
dkropachev merged commit dcb16da into scylladb:master Oct 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test run 2026.1.15 failed with error: "FileNotFoundError: [Errno 2] No such file or directory"

1 participant