Conversation
luhenry
pushed a commit
that referenced
this pull request
Sep 19, 2026
Add build-vllm.yml and docs/packages/vllm.yaml for vLLM 0.29.0+cpu, the VLLM_TARGET_DEVICE=cpu build of vllm-project/vllm. vLLM's CPU backend already has first-class riscv64 support upstream: a riscv64 branch in cmake/cpu_extension.cmake, RVV kernels under csrc/cpu (cpu_types_riscv*.hpp, sgl-kernels/gemm_int4.cpp, cpu_wna16.cpp), a CpuArchEnum.RISCV platform, and platform_machine == "riscv64" markers in requirements/cpu.txt that already pin torch==2.13.0 - exactly the version this registry publishes as 2.13.0+cpu. The wheel is built at the rv64gc baseline (-DVLLM_RVV_VLEN=0, upstream's documented scalar RISC-V mode) rather than letting cmake auto-detect VLEN from the runner's /proc/cpuinfo, which would bake that runner's vector extensions into a wheel shipped to every riscv64 user.
A GPU package whose PyPI wheels are unconditionally CUDA can still build a portable CPU distribution from the same tree when setup.py selects the device from an env var. Records how to tell the two apart, why upstream riscv64 requirement markers are the fastest evidence, when an absent optional dependency is a marker patch rather than a park, and why a native riscv64 runner's /proc/cpuinfo must not drive a published wheel's -march.
cibuildwheel's before-build cwd is not guaranteed to be the package directory when package-dir points at a subdirectory, so anchor the requirements path explicitly.
luhenry
changed the title
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
vllm0.29.0+cpuCompiles vLLM's CPU inference backend (
vllm._Cand the PyO3 frontend) against thetorch2.13.0+cpu wheels this registry already publishes. Upstream publishes no riscv64 wheel.Mirrors upstream's
docker/Dockerfile.cpu, vLLM's only CPU wheel build.Differs from upstream
-DVLLM_RVV_VLEN=0- auto-detection would bake the runner's own VLEN into every wheel.Matrix: one
cp312build.CMakeExtensionsetspy_limited_api, so upstream ships a singlecp38-abi3wheel for every interpreter and so do we. cp312 is also the floor at whichtorch==2.13.0resolves on riscv64.Testing: imports
vllmandvllm._Cfrom the installed wheel and reports the detected CPU architecture; upstream's own suites need model downloads.License: OK
Patches
0001-requirements-do-not-require-numba-on-riscv64.patch-Upstream-Status: To upstream. Without it the wheel is buildable but not installable:requirements/cpu.txtrequires numba everywhere but s390x, and numba has no riscv64 wheel and cannot be built from sdist (llvmlite needs a conda-only patched LLVM with no linux-riscv64 subdir). numba is optional to vLLM by construction (is_numba_available()), and this is the configuration upstream already ships for s390x. riscv64-only.Not yet built:
quay.ioandpypi.riseproject.devare both unreachable from the authoring environment, so nothing was rehearsed locally. Draft until CI produces a wheel.