Skip to content

fix(sources): prepare Git checkouts by download kind - #1389

Merged
LalatenduMohanty merged 1 commit into
python-wheel-build:new-resolver-configfrom
smoparth:codex/issue-1205-prepare-git-checkout
Oct 23, 2026
Merged

LalatenduMohanty merged 1 commit into
python-wheel-build:new-resolver-configfrom
smoparth:codex/issue-1205-prepare-git-checkout

Conversation

@smoparth

@smoparth smoparth commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

  • Pass the downloaded artifact kind through source preparation. Archives
    are unpacked; clean Git checkouts are prepared in place.
  • Keep existing prepare_source override arguments and path-only callers
    compatible. Build metadata is written but is not used to decide
    whether preparation runs.
  • Add --source-kind to step prepare-source and optional JSON output
    to step download-source-archive. Existing path-only output remains
    the default.
  • Replace an existing checkout when cleanup is enabled. Without cleanup,
    an existing destination raises FileExistsError.

Why

Git resolvers return checkout directories, but preparation previously
tried to unpack those directories as archives.

Closes: #1205

The related build_sdist strategy remains in #1206 / #1385. The current
default tar builder can include .git, so Git source builds are not yet
production-ready. Prebuilt routing remains in #1208.

Testing

  • Focused tests in test_sources.py, test_packagesettings_resolver.py,
    test_bootstrapper.py, and test_commands.py: 244 passed.

  • File-scoped Ruff and mypy checks passed; git diff --check passed.

  • Full and end-to-end suites were not run.

  • PR follows CONTRIBUTING.md guidelines

@smoparth
smoparth requested a review from a team as a code owner October 22, 2026 20:28
@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mergify mergify Bot added the ci label Oct 22, 2026

@rd4398 rd4398 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks overall good. The tests and docs are thorough too. I have left some comments

Comment thread src/fromager/sources.py Outdated
Comment thread src/fromager/sources.py Outdated
Comment thread src/fromager/packagesettings/_resolver.py Outdated
Comment thread src/fromager/commands/step.py
Comment thread src/fromager/commands/step.py Outdated
Comment thread docs/reference/hooks.rst
Comment thread tests/test_commands.py Outdated
@mergify

mergify Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be merged.
@smoparth please rebase your branch.

@smoparth
smoparth force-pushed the codex/issue-1205-prepare-git-checkout branch from 978129b to 8a4057c Compare October 23, 2026 16:25
rd4398
rd4398 previously approved these changes Oct 23, 2026

@rd4398 rd4398 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the feedback. This looks good now!

@mergify

mergify Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

@LalatenduMohanty LalatenduMohanty left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--source-path can cause Fromager to delete the input archive. For example, passing work-dir/demo-1.0/demo-1.0.tar.gz makes unpack_source() delete its containing directory before opening the archive. The command then fails with FileNotFoundError, and the archive is gone.

The cleanup code already exists, but this PR exposes it through the new option. Please reject this location before cleanup and add a test confirming that the archive survives.

Proposed diff in src/fromager/sources.py

@@ -308,5 +308,16 @@
     unpack_dir = ctx.work_dir / expected_name
+    resolved_unpack_dir = unpack_dir.resolve()
+    source_paths = (source_filename, *source_filename.parents)
+    if any(
+        path.resolve().is_relative_to(resolved_unpack_dir)
+        for path in source_paths
+    ):
+        raise ValueError(
+            f"source archive {source_filename} overlaps {unpack_dir}; "
+            "move it outside the package work directory before preparing"
+        )
+
     if unpack_dir.exists():
         if ctx.cleanup:
             logger.debug("cleaning up %s", unpack_dir)
             shutil.rmtree(unpack_dir)

The dangerous line is shutil.rmtree(unpack_dir): it deletes the folder and everything inside it.

The added check runs before deletion. It checks the archive and its parent folders, resolving symlinks, and stops with a clear error if they overlap the folder being cleaned.

The proposed guard passed eight focused path checks. It still needs a regression test in the PR.

Carry DownloadedSource.kind through bootstrap and build entry points. Unpack ordinary archives; prepare clean Git checkouts in place so patches, project overrides, and Rust vendoring run.

Keep prepare_source override arguments and path-only callers compatible. Document the public Git helper and no-cleanup behavior, and reject dirty nested submodules before preparation.

Reject archive input paths that cleanup would remove before extraction, preserving the input and reporting an actionable error.

Closes: python-wheel-build#1205
Co-Authored-By: Codex <noreply@openai.com>
Signed-off-by: Shanmukh Pawan <smoparth@redhat.com>
@smoparth
smoparth force-pushed the codex/issue-1205-prepare-git-checkout branch from 8a4057c to c5e3a28 Compare October 23, 2026 18:21
@LalatenduMohanty
LalatenduMohanty merged commit a7bc164 into python-wheel-build:new-resolver-config Oct 23, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants