Repository navigation
RFC: pkgx.sh/bootstrap-tools — seed bottle to close pantry's bootstrap problem #12972
Description
Status: RFC / proposal
Asks for buy-in on creating a new top-level package (pkgx.sh/bootstrap-tools) and a companion build-infra repo (pkgxdev/bootstrap-tools) that, together, would close the bootstrap problem for pantry's GNU toolchain (the chicken-and-egg of building old gcc when only modern gcc exists in dist.pkgx.dev).
This is companion-piece #4 to the structural improvements proposed in:
- fix-elf: patchelf PT_INTERP to strip +brewing post-install brewkit#342 — fix-elf PT_INTERP patching
- useShellEnv: don't auto-export lib/ for libc-style bottles libpkgx#85 — libc-bottle env-export opt-out
- feat(build-script):
linux-sysroot:directive for host-independent builds brewkit#343 —build.sysroot:directive
Items 1–3 are small additive changes. This proposal (#4) is larger structurally and warrants discussion before code.
The bootstrap problem we hit
In #12968 (gnu.org/glibc) and the companion older-toolchain PRs (#12966 binutils, #12967 gcc), we demonstrated that pantry CAN build the HPC range — glibc 2.17 (CentOS 7 / manylinux2014 baseline) through 2.43 (current) on both linux/x86-64 and linux/aarch64. The catch: each cascade step needs an older compiler than the one pkgx ships. We built it manually:
pkgx gcc 16 (existing dist.pkgx.dev)
↓ builds glibc 2.43
↓ builds gcc 9.5 (using gcc 16 + glibc 2.43 sysroot)
↓ builds binutils 2.32 (using gcc 9.5)
↓ builds gcc 7.5 (using gcc 9.5)
↓ builds binutils 2.28 (using gcc 7.5)
↓ builds glibc 2.24 + glibc 2.17 (using gcc 7.5 + binutils 2.28)
Each step takes 30–60 min. The full chain is multi-hours. Done once, the resulting bottles serve everyone, forever — but right now there's no place for them to live in dist.pkgx.dev and no way for brewkit CI to chain them.
Proposal
Two new artefacts:
1. pkgxdev/bootstrap-tools (new repo, build infrastructure)
A repo with:
- A GitHub Actions workflow that builds a single "seed" tarball on a
centos:7(or equivalent) base image with naturally-old glibc 2.17 — call it the bootstrap origin - Output:
bootstrap-tools-<version>-{linux+x86-64,linux+aarch64}.tar.xzpublished to GitHub Releases - Contents (statically linked or
patchelf $ORIGIN-relocatable):
bootstrap-tools/v1.0.0/
├── bin/
│ ├── gcc, g++, gfortran # gcc ~9.5
│ ├── ld, as, ar, ranlib, nm, objdump, strip, ... # binutils ~2.32
│ ├── make, m4, awk, sed, grep, sort, ...
│ └── sh, bash, perl, python
├── lib/
│ ├── libc.so.6 ld-linux-*.so.* # glibc 2.17 (manylinux2014 baseline)
│ ├── libstdc++.so.*, libgcc_s.so.*
│ └── ...
├── include/
└── relocate.sh # post-extract patchelf to RPATH=$ORIGIN
Size estimate: ~80 MB compressed, ~250 MB uncompressed.
2. pkgx.sh/bootstrap-tools (pantry recipe)
A tiny recipe that downloads + extracts the tarball — NO compiler needed:
distributable:
url: https://github.com/pkgxdev/bootstrap-tools/releases/download/v{{version}}/bootstrap-tools-{{version}}-{{hw.platform}}+{{hw.arch}}.tar.xz
strip-components: 1
versions:
github: pkgxdev/bootstrap-tools
platforms: linux
build:
script:
- cp -a . {{prefix}}/
- {{prefix}}/relocate.sh "{{prefix}}"
test:
- {{prefix}}/bin/gcc --version | grep -q "GCC"
- {{prefix}}/bin/ld --version | grep -q "GNU"
- {{prefix}}/lib/ld-linux-*.so.* --versionUsage by old-version recipes
The HPC cascade becomes expressible in pantry yml:
# Older glibc, e.g. gnu.org/glibc/v2.17.0
build:
sysroot:
libc: pkgx.sh/bootstrap-tools # routes gcc at bootstrap's glibc 2.17
dependencies:
pkgx.sh/bootstrap-tools: '*'
# gcc and binutils from bootstrap-tools' bin/, not the modern dist.pkgx.dev onesCombined with #343's build.sysroot: directive, every recipe in the pantry could optionally target glibc 2.17 for manylinux2014 ABI baseline. Big leap toward Nix nixpkgs parity.
Why this needs maintainer buy-in (not just a PR)
- New top-level project namespace (
pkgx.sh/...) — only pkgxdev should own that. - Build infrastructure on
centos:7— needs a runner that can use that image, and a workflow that pushes Releases underpkgxdev/bootstrap-tools. - One-time blessing of binary blobs — the bootstrap blob is binary, but it's GPL-compatible (gcc, glibc, binutils, coreutils, perl, python). Same legal/policy bar as nixpkgs' bootstrap-tools.
- Long-term commit — once recipes depend on it, the format becomes a stable API.
What we have if you want to start
I have working build scripts (from session 2026-05-19/20 in pkgm/notes/) that produce the cascaded gcc 7.5/9.5 + binutils 2.28 inside Docker containers. They could be the seed of pkgxdev/bootstrap-tools workflows. Happy to:
- Open a starter repo with the GitHub Actions workflow that produces the tarball deterministically
- Run it locally once to verify reproducibility cross-arch (linux/x86-64 + linux/aarch64)
- Open the
pkgx.sh/bootstrap-toolsrecipe PR alongside
…but only after this RFC has direction from a pkgxdev maintainer.
Smaller stepping-stone alternative
If the bootstrap-tools blob is too big a first step, an interim solution: publish only the cascaded gcc 7.5/9.5 + binutils 2.28 bottles built via #12967 + #12966 (when merged) to dist.pkgx.dev as normal pantry artefacts. Then a pantry recipe gnu.org/glibc=~2.17 could use build.sysroot: + those as deps. This skips the "blessed seed blob" entirely at the cost of a longer chain.
I have no strong preference between the two paths — both are improvements. Curious which the maintainers prefer.
Refs
- feat(gnu.org/glibc): host-independent recipe, 9 versions × 2 arches incl. HPC 2.17 / 2.24 #12968 (gnu.org/glibc) + companions — demonstration of the cascade in action
- feat(build-script):
linux-sysroot:directive for host-independent builds brewkit#343 —build.sysroot:directive (paired) - platform support megaticket pkgx#607 — platform-support megaticket
- pkgm/notes/session-2026-05-{19,20}.md — full empirical record
- nixpkgs' bootstrap-tools — same concept in their ecosystem
🤖 Generated with Claude Code
Activity
tannevaled commentedon May 20, 2026
Update: starter implementation pushed
I've put a draft implementation at tannevaled/bootstrap-tools for review.
Contents:
Dockerfile(multi-stage onquay.io/pypa/manylinux2014_*)build.sh(5-stage cascade: binutils 2.32 → gcc 9.5 → bundle glibc 2.17 runtime → patchelf → smoke-test)relocate.sh($ORIGIN-relative RPATH patcher for post-extract relocation).github/workflows/release.yml(matrix build x86-64 + aarch64 on tagv*, uploads to Release)docs/cascade.md(the procedure documented end-to-end)README.md,LICENSE.txt(GPL-3.0)Pinned versions: gcc 9.5.0 + binutils 2.32 + glibc 2.17 (the last reused as-is from the manylinux2014 base image — it IS the ABI baseline we want to ship).
The starter has NOT been CI-tested end-to-end yet (no released tag → no workflow run). Logical next step would be a maintainer-blessed CI run on
pkgxdev/bootstrap-tools(if the proposal gets traction) to validate the build chain produces a working tarball.I'm happy to:
pkgx.sh/bootstrap-toolsrecipe PR on pantry once the seed tarball format stabilises…whichever order helps. Awaiting direction.
jhheider commentedon May 20, 2026
well, getting glibc built is certainly desired. i like this idea, but i'm not sure i saw why it needs a bootstrap pantry. if we can get glibc built, would it not be sufficient to have brewkit load in 2.17 unless overridden by a different version (the way we do now for compilers)? that would (potentially) involve some logic on pkgxdev/pkgx to load it at runtime, maybe, except binaries usually run on later glibc (iirc), so any that required specific versions could dependency them as normal, right?
tannevaled commentedon May 21, 2026
@jhheider — fair, and on reflection your simpler framing is the right one. Let me unpack why I jumped to "bootstrap pantry" and then converge on what you proposed:
Why I reached for a bootstrap pantry
The cold-start problem is: to build a glibc N.M, you need a working gcc + binutils + glibc on the build host. Pantry's own gcc/binutils tarballs already statically link or RPATH-resolve their own libc, so we almost don't have a chicken-and-egg. The only real gap was that gcc 16 + glibc 2.17 don't compile together cleanly without
-fcommon -Wno-errorand alibc_cv_static_pie=nodance (both now handled in #12968). So once #12966 + #12967 + #12968 land, building 2.17 from adebian:bookworm-slimhost with no apt-installed compiler works — I verified this empirically.That makes a separate bootstrap tarball optional, not load-bearing.
Your proposal: brewkit ships a default glibc, packages can override
This is much cleaner. Concretely it looks like:
useShellEnvpicksgnu.org/glibc@2.17as the default sysroot for any Linux build (the way it picks a defaultgcc).gnu.org/glibc: '>=2.34'inbuild.dependencies— same dispatch as compilers today.And re: your "binaries usually run on later glibc": correct — glibc is forward-compatible at the symbol level via versioned symbols (
__libc_start_main@GLIBC_2.17), so a binary linked against 2.17 runs on any host ≥ 2.17. That's literally the manylinux2014 contract. So defaulting to 2.17 as the baseline is the same compatibility envelope manylinux uses, and it's the right floor.Where bootstrap-tools still has a role (if at all)
Only if pkgxdev/pkgx itself ever wants a "cold start" tarball with no compiler in
$PATH— i.e. the very first install of pkgx on a machine with literally nothing pre-installed. That's a fringe case; today pkgx assumes a host with at least a glibc and a dynamic linker present.Suggested path forward
tannevaled/bootstrap-toolsas an example/reference (the Dockerfile + cascade is documented and CI-green on both arches — useful for "how do you bake a glibc-2.17 toolchain from scratch" if anyone ever needs it).gnu.org/glibc@2.17as default Linux sysroot, allow per-package override via build.dependencies".build.sysroot.libcdirective from brewkit#343 becomes the explicit-override mechanism for packages that want a different version.WDYT — want me to retitle this issue to track item #3 above, or close it and open a fresh brewkit issue?
jhheider commentedon May 21, 2026
Use whatever pr/issues you like. I think the easy way is:
tannevaled commentedon May 21, 2026
Great, opened pkgxdev/brewkit#344 to track the hermetic-glibc helper (
bklibcvenv, analogue tobkpyvenv) as a separate, optional layer on top of your default-glibc plan.Mapping how the four pieces compose:
useShellEnv(your proposal)build.dependencies.gnu.org/glibc: '>=X.Y'build.skip: linux-glibcbklibcvenv stage/seal(pkgxdev/brewkit#344)I'll leave this RFC open until your default-glibc piece lands, then close it pointing at #344. The bootstrap-tools repo at
tannevaled/bootstrap-toolsstays as a reference artifact in case the cold-start case ever comes up — but it's not on the critical path anymore.Re your last reply ("Use whatever pr/issues you like") — happy to:
useShellEnvdefault-glibc-injection in brewkit (probably the smallest change of the three layers); orWhichever ordering you prefer.
tannevaled commentedon May 29, 2026
Closing: the bootstrap problem this RFC raised has been solved without needing a separate
pkgx.sh/bootstrap-toolspantry.What actually shipped (May 2026):
The "we need a separate bootstrap seed" framing turned out to be unnecessary — pantry's own cascade is comprehensive enough. Older toolchain bottles (gcc 7.5 / binutils 2.28) build cleanly inside pantry itself using newer host toolchains, given the
-fcommon -Wno-error+libc_cv_static_pie=noworkarounds we landed in #12968.The
tannevaled/bootstrap-toolsrepo stays as a reference for the manylinux2014-Docker-based bootstrap approach (useful if pkgx ever needs cold-start without ANY compiler), but isn't on the critical path.@jhheider thanks for steering this toward the simpler "brewkit just loads 2.17 by default" framing back in the original thread — that was the right call.