LCORE-2874: Migrate to RHOAI 3.4 + PyPI - main#2023
Conversation
WalkthroughThis PR introduces a new policy-driven dependency resolution script (konflux_resolve.py) and supporting Konflux configuration files, regenerates requirements/hash/override files targeting RHOAI 3.4, upgrades base images and RPM packages (gnutls, libxslt, openssl-fips-provider), updates Tekton prefetch package lists, switches Containerfile build steps to use uv, and bumps torch in pyproject.toml. ChangesKonflux Resolution and Build Infrastructure Update
Sequence Diagram(s)sequenceDiagram
participant Developer
participant konflux_resolve_main as Main
participant uv
participant RhoaiIndex
participant PypiClient
participant Tekton as TektonYAML
Developer->>Main: run konflux_resolve.py --profile cpu
Main->>uv: uv pip compile --index-strategy prefer-index
uv-->>Main: resolved versions + index annotations
Main->>RhoaiIndex: reclassify_with_rhoai(resolved)
RhoaiIndex-->>Main: RHOAI wheels matched
Main->>PypiClient: fetch hashes for remaining PyPI packages
PypiClient-->>Main: sha256 hashes
Main->>Main: write_hashed_requirements (source/wheel/pypi)
Main->>Tekton: patch_tekton_packages(package_names)
Main-->>Developer: print resolution summary
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
/retest |
syedriko
force-pushed
the
syedriko-rhoai-3.4-main
branch
from
08928c9 to
c7171c2
Compare
|
/retest |
syedriko
force-pushed
the
syedriko-rhoai-3.4-main
branch
from
c7171c2 to
f0176e3
Compare
syedriko
force-pushed
the
syedriko-rhoai-3.4-main
branch
from
f0176e3 to
ccc3010
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.tekton/lightspeed-stack-0-6-pull-request.yaml (1)
35-63: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winAdd
maturinto the prefetch inputs —binary.packagesincludesmaturin, but it is missing from all of the referenced hash files, so the hermetic prefetch set is incomplete. If it is no longer needed, remove it frombinary.packagesinstead.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.tekton/lightspeed-stack-0-6-pull-request.yaml around lines 35 - 63, The prefetch configuration in the lightspeed stack is incomplete because `maturin` is listed in `prefetch-input` under `binary.packages` but is missing from the referenced requirements hash files. Update the `prefetch-input` entry so `maturin` is either added to the appropriate hash/requirements files used by this stack or removed from `binary.packages` if it is no longer needed, keeping the `prefetch-input` list consistent with the hermetic dependency set.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/generate-rpm-lock.sh`:
- Around line 5-33: The fallback base image in generate-rpm-lock.sh is still
using the old registry.redhat.io/rhai/base-image-cpu-rhel9:3.4 value, so update
the DEFAULT_BASE_IMAGE constant to the migrated Konflux image used by this PR.
Make sure the logic in the build-args fallback path continues to prefer
BUILDER_BASE_IMAGE from .konflux/build-args-konflux.conf, but when it is
missing, BASE_IMAGE should resolve to the new
quay.io/aipcc/base-images/cpu:3.4.2-1782270165 image instead of the legacy one.
- Around line 108-109: The repo disable step in generate-rpm-lock.sh is
hardcoded to x86_64 and will fail on non-x86_64 containers. Update the
repository handling around the subscription-manager repos call to derive the
repo IDs from the container architecture (or check availability before
disabling) instead of unconditionally disabling rhel-9-for-x86_64-* entries.
Keep the existing echo/logging, but make the disable logic architecture-aware so
the script works for aarch64 as well.
In `@scripts/konflux_resolve.py`:
- Around line 447-471: The package resolution flow in the wheel selection logic
only keeps a version when any target arch matches, which can miss incomplete
wheel sets for other configured architectures. Update the resolution path in the
parser/collector and the version selection in find_best() so all configured
target architectures from profiles.toml must be satisfied before accepting a
release, and ensure the stored package data tracks per-arch wheel availability
consistently across the affected code paths.
- Around line 548-591: The marker handling in _eval_single_marker and the
surrounding marker parser is doing string-based comparisons and defaulting
unparsed expressions to True, which can produce incorrect results. Replace this
logic with a real PEP 508 marker evaluator, or at minimum make _MARKER_CMP_OPS
and _eval_single_marker version-aware for fields like python_version and
platform values. Ensure unsupported or malformed markers are not silently
accepted as True, and keep the existing marker evaluation entry point so the fix
is localized.
- Around line 1274-1283: Step 7 is hardcoding the uv executable instead of
reusing the resolved binary path. Update the pybuild-deps compile call in
uv_resolve() to use uv_resolved, which already handles $UV_BINARY and the
repo-local fallback, so requirements-build.txt generation works on all hosts.
Keep the change localized to the subprocess.run invocation in uv_resolve().
---
Outside diff comments:
In @.tekton/lightspeed-stack-0-6-pull-request.yaml:
- Around line 35-63: The prefetch configuration in the lightspeed stack is
incomplete because `maturin` is listed in `prefetch-input` under
`binary.packages` but is missing from the referenced requirements hash files.
Update the `prefetch-input` entry so `maturin` is either added to the
appropriate hash/requirements files used by this stack or removed from
`binary.packages` if it is no longer needed, keeping the `prefetch-input` list
consistent with the hermetic dependency set.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: a9272d86-0d7e-4606-8ad7-5a58861debc8
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (19)
.konflux/build-args-konflux.conf.konflux/profiles.toml.konflux/pypi_wheel_only.txt.konflux/requirements-build.txt.konflux/requirements.hashes.source.txt.konflux/requirements.hashes.wheel.pypi.txt.konflux/requirements.hashes.wheel.txt.konflux/requirements.hermetic.txt.konflux/requirements.overrides.txt.konflux/rpms.in.yaml.konflux/rpms.lock.yaml.tekton/lightspeed-stack-0-6-pull-request.yaml.tekton/lightspeed-stack-0-6-push.yaml.tekton/lightspeed-stack-pull-request.yaml.tekton/lightspeed-stack-push.yamldeploy/lightspeed-stack/Containerfilepyproject.tomlscripts/generate-rpm-lock.shscripts/konflux_resolve.py
💤 Files with no reviewable changes (1)
- .konflux/requirements.hermetic.txt
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-on-pull-request
- GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-6-on-pull-request
⚠️ CI failures not shown inline (4)
GitHub Actions: OpenAPI (Spectral) / 0_spectral.txt: Migrate to RHOAI 3.4 + PyPI - main
Conclusion: failure
##[group]Run set -euo pipefail
�[36;1mset -euo pipefail�[0m
�[36;1muv run python scripts/generate_openapi_schema.py /tmp/openapi-generated.json�[0m
�[36;1mif ! diff -u docs/openapi.json /tmp/openapi-generated.json; then�[0m
�[36;1m echo "::error::docs/openapi.json is out of date. Regenerate with: uv run scripts/generate_openapi_schema.py docs/openapi.json"�[0m
GitHub Actions: Unit tests / 1_unit_tests (3.12).txt: Migrate to RHOAI 3.4 + PyPI - main
Conclusion: failure
##[group]Run uv run pytest tests/unit --cov=src --cov=runner --cov-report term-missing
�[36;1muv run pytest tests/unit --cov=src --cov=runner --cov-report term-missing�[0m
shell: /usr/bin/bash -e {0}
env:
UV_PYTHON: 3.12
VIRTUAL_ENV: /home/runner/work/lightspeed-stack/lightspeed-stack/.venv
UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
##[endgroup]
Uninstalled 1 package in 2ms
Installed 1 package in 12ms
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/runner/work/lightspeed-stack/lightspeed-stack
configfile: pyproject.toml
plugins: asyncio-1.4.0, benchmark-5.2.3, anyio-4.14.1, order-1.5.0, mock-3.15.1, cov-7.1.0, logfire-4.37.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 2928 items
tests/unit/a2a_storage/test_in_memory_context_store.py ........ [ 0%]
tests/unit/a2a_storage/test_sqlite_context_store.py .......... [ 0%]
tests/unit/a2a_storage/test_storage_factory.py ........... [ 0%]
tests/unit/app/endpoints/test_a2a.py .............................. [ 2%]
tests/unit/app/endpoints/test_authorized.py ... [ 2%]
tests/unit/app/endpoints/test_config.py .. [ 2%]
tests/unit/app/endpoints/test_conversations.py ......................... [ 3%]
................. [ 3%]
tests/unit/app/endpoints/test_conversations_v2.py ...................... [ 4%]
............... [ 4%]
tests/unit/app/endpoints/test_feedback.py ....................... [ 5%]
tests/unit/ap...
GitHub Actions: Unit tests / 0_unit_tests (3.13).txt: Migrate to RHOAI 3.4 + PyPI - main
Conclusion: failure
##[group]Run uv run pytest tests/unit --cov=src --cov=runner --cov-report term-missing
�[36;1muv run pytest tests/unit --cov=src --cov=runner --cov-report term-missing�[0m
shell: /usr/bin/bash -e {0}
env:
UV_PYTHON: 3.13
VIRTUAL_ENV: /home/runner/work/lightspeed-stack/lightspeed-stack/.venv
UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
##[endgroup]
Uninstalled 1 package in 2ms
Installed 1 package in 3ms
============================= test session starts ==============================
platform linux -- Python 3.13.14, pytest-9.1.1, pluggy-1.6.0
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/runner/work/lightspeed-stack/lightspeed-stack
configfile: pyproject.toml
plugins: asyncio-1.4.0, benchmark-5.2.3, anyio-4.14.1, order-1.5.0, mock-3.15.1, cov-7.1.0, logfire-4.37.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 2928 items
tests/unit/a2a_storage/test_in_memory_context_store.py ........ [ 0%]
tests/unit/a2a_storage/test_sqlite_context_store.py .......... [ 0%]
tests/unit/a2a_storage/test_storage_factory.py ........... [ 0%]
tests/unit/app/endpoints/test_a2a.py .............................. [ 2%]
tests/unit/app/endpoints/test_authorized.py ... [ 2%]
tests/unit/app/endpoints/test_config.py .. [ 2%]
tests/unit/app/endpoints/test_conversations.py ......................... [ 3%]
................. [ 3%]
tests/unit/app/endpoints/test_conversations_v2.py ...................... [ 4%]
............... [ 4%]
tests/unit/app/endpoints/test_feedback.py ....................... [ 5%]
tests/unit/ap...
GitHub Actions: PR Title Checker / 0_check.txt: Migrate to RHOAI 3.4 + PyPI - main
Conclusion: failure
##[group]Run thehanimo/pr-title-checker@v1.4.3
with:
GITHUB_***REDACTED***
pass_on_octokit_error: false
configuration_path: .github/pr-title-checker-config.json
##[endgroup]
(node:2162) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
Using config file .github/pr-title-checker-config.json from repo lightspeed-core/lightspeed-stack [ref: 56ae5fdc6bce2da7499bfeffad1c0c30baf32c8e]
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:2162) [DEP0169] DeprecationWarning: `url.parse()` behavior is not standardized and prone to errors that have security implications. Use the WHATWG URL API instead. CVEs are not issued for `url.parse()` vulnerabilities.
Creating label (title needs formatting)...
Label (title needs formatting) already created.
Adding label (title needs formatting) to PR...
HttpError: Resource not accessible by integration
##[error]Failed to add label (title needs formatting) to PR
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2026-05-12T15:14:34.788Z
Learnt from: syedriko
Repo: lightspeed-core/lightspeed-stack PR: 1727
File: scripts/konflux_requirements.sh:9-15
Timestamp: 2026-05-12T15:14:34.788Z
Learning: In this repo, the `.konflux/` directory is committed/tracked and is guaranteed to exist in a fresh clone. Therefore, shell scripts that write output under `.konflux/` (e.g., create files like `.konflux/<...>`) should not waste effort by calling `mkdir -p .konflux` first. Only add directory-creation logic if the script may run in an environment/repo state where `.konflux/` might not be present.
Applied to files:
scripts/generate-rpm-lock.sh
📚 Learning: 2026-06-24T13:45:37.249Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 1971
File: src/utils/markdown_repair.py:31-36
Timestamp: 2026-06-24T13:45:37.249Z
Learning: In the lightspeed-stack repository, docstrings must use the section header name "Parameters:" (not "Args:") for function arguments, even if the project references Google Python docstring conventions. Ensure docstrings follow the project’s established "Parameters:" header format for any documented function parameters.
Applied to files:
scripts/konflux_resolve.py
🪛 ast-grep (0.44.0)
scripts/konflux_resolve.py
[error] 1040-1040: Use of unsanitized data to create processes
Context: subprocess.run(cmd, capture_output=True, text=True, check=True)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(os-system-unsanitized-data)
[warning] 204-204: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(pyproject_path, "rb")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 914-914: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(output_path, "w")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 925-925: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(yaml_path)
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 932-932: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(yaml_path, "w")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 950-950: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(profiles_path, "rb")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 972-972: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(path)
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 1151-1151: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(build_file)
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 1166-1166: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(build_file, "w")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 1269-1269: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(tmp_sdist_file, "w")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 1293-1293: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(build_output, "w")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[warning] 417-417: Request-controlled URL passed to urlopen; validate against an allowlist to prevent SSRF.
Context: urllib.request.urlopen(url, timeout=30)
Note: [CWE-918] Server-Side Request Forgery (SSRF).
(urlopen-unsanitized-data)
[warning] 609-609: Request-controlled URL passed to urlopen; validate against an allowlist to prevent SSRF.
Context: urllib.request.urlopen(url, timeout=30)
Note: [CWE-918] Server-Side Request Forgery (SSRF).
(urlopen-unsanitized-data)
[error] 1040-1040: Command coming from incoming request
Context: subprocess.run(cmd, capture_output=True, text=True, check=True)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
[error] 1273-1283: Command coming from incoming request
Context: subprocess.run(
[
"uv",
"run",
"pybuild-deps",
"compile",
f"--output-file={build_output}",
tmp_sdist_file,
],
check=True,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
🔇 Additional comments (11)
pyproject.toml (2)
189-189: LGTM!
241-243: 📐 Maintainability & Code Quality
[tool.black]is already wired into the workflow The repo runs Black viaMakefileand.github/workflows/black.yaml, so this config isn’t dead or duplicate tooling.> Likely an incorrect or invalid review comment..konflux/rpms.lock.yaml (1)
49-55: LGTM!Also applies to: 91-118, 165-171, 207-234
.konflux/rpms.in.yaml (1)
10-15: 🗄️ Data Integrity & IntegrationNo change needed for
upgradePackagesrpm-lockfile-prototypesupports this input key, and it uses the same list schema aspackages.> Likely an incorrect or invalid review comment..konflux/pypi_wheel_only.txt (1)
1-5: 🩺 Stability & AvailabilityLeave
.konflux/pypi_wheel_only.txtempty. The binary-heavy packages are already landing in.konflux/requirements.hashes.wheel.txt, not the PyPI wheel bucket..konflux/requirements-build.txt (1)
5-48: LGTM!.konflux/requirements.hashes.source.txt (1)
2-23: LGTM!.konflux/requirements.hashes.wheel.pypi.txt (1)
1-2: LGTM!.konflux/requirements.overrides.txt (1)
2-3: LGTM!.konflux/requirements.hashes.wheel.txt (1)
1-443: 🎯 Functional CorrectnessDrop the
maturinhash-file concern.maturinis handled as a bootstrap/extra wheel (EXTRA_WHEELSandbootstrap_packages), so it does not need an entry in.konflux/requirements.hashes.wheel.txt.> Likely an incorrect or invalid review comment..tekton/lightspeed-stack-0-6-push.yaml (1)
36-64: 🎯 Functional Correctness
binary.packagesis already aligned with the pull-request pipeline; the only open point is whether.konflux/requirements.hermetic.txtis an expected generated artifact for this target.
syedriko
changed the title
1 participant
Description
Migrate to RHOAI 3.4 + PyPI on the main branch in preparation for 0.6 release:
Type of change
Tools used to create PR
Identify any AI code assistants used in this PR (for transparency and review context)
Related Tickets & Documents
Checklist before requesting a review
Testing