feat(bb): publish bb and bb-avm as npm packages, one per platform, byte-identical to the release tarballs - #25356
Open
ludamad wants to merge 29 commits into
Open
feat(bb): publish bb and bb-avm as npm packages, one per platform, byte-identical to the release tarballs#25356ludamad wants to merge 29 commits into
ludamad wants to merge 29 commits into
Conversation
…ed publishing The EC2 release build no longer publishes to npm. deploy_npm gains a pack mode (NPM_PACK_DIR): the same version stamping and workspace rewriting as before, but the result is an `npm pack` tarball dropped into npm-release/. bootstrap.sh release uploads that directory to the build cache as npm-release-<tag>.tar.gz. A new publish-npm job in ci3.yml (tags only, public repo only, `release` environment, id-token: write) downloads the bundle and runs ci3/publish_npm_release, which publishes each tarball with `--provenance` via npm trusted publishing. No NPM_TOKEN is involved on either side; the build machine never holds publish rights. The private repo is untouched: its private_release keeps publishing to the internal GCP Artifact Registry from EC2.
…y in SSM mode Port of aztec-packages-private#713: SSM mode authenticates to the build instance with an instance profile, so BUILD_INSTANCE_SSH_KEY is only written on the direct-SSH path (CI_USE_SSH=1). With no key on the runner, source_redis skips the bastion tunnel instead of failing the run, and denoise/source_cache cope with redis being absent. A rotated or missing bastion key can no longer take CI down. The build log stays reachable the way it always was: bootstrap_ec2 posts the ci/<job> commit status with the dashboard link, and the instance logs to redis from inside the VPC. Two small fixes so that holds without redis on the runner: denoise prints its captured output when it has no log store and CI=1 (the orchestration output, with the CI Log link, used to vanish on success), and cache_log persists the instance's top-level log to S3 synchronously (the background upload was killed with the container before it landed).
…scope Every npm package this repository publishes moves from @aztec/* to @aztec-foundation/*: bb.js, bb-avm-sim (+ platform packages), cdb, ipc-runtime, wsdb (+ platform packages), the noir packages (@noir-lang/* now rename to @aztec-foundation/noir-* at build and release time), l1-artifacts, constants-codegen and the three protocol artifacts packages. The scope is the one npm trusted publishing is being configured for; the labs packages keep @Aztec and are published from the labs repository. Consumers inside this tree (portals, imports, generator defaults, scripts, comments) follow, and the affected yarn lockfiles are refreshed for the renamed workspace entries only. The labs submodule keeps importing @aztec/bb.js etc.: a yarn portal resolves by path and accepts a target manifest with a different name, so use-local keeps working until labs moves to the new names. The private release now scopes @aztec-foundation to the internal registry.
…te-identical to the release tarballs
The bb binary inside @aztec/bb.js could report a version days older than the
package: bb.js restores its dest/ from a cache keyed on source, not on the
release, so an unchanged tree resurrected an earlier build with the version
stamped back then; and the release tarballs were unstripped while bb.js's
copies were stripped, so no two channels shipped the same bytes.
Now every distributed bb/bb-avm goes through one step, finalize_bb_binary in
barretenberg/cpp/bootstrap.sh (strip, then inject_version, which re-signs
Mach-O), used by the release tarballs (release_tar), by bb.js's bundled copies
at release time, and by the new npm packages:
- @aztec-foundation/bb — a meta package with the `bb` command and optional
dependencies on bb-{linux,darwin}-{x64,arm64} and bb-win32-x64;
- @aztec-foundation/bb-avm — the same shape over bb-avm-linux-{x64,arm64}.
They are plain packable directories under barretenberg/ts/bb-cli and
bb-avm-cli (no workspace, no lockfile); scripts/native_packages.sh stages the
binaries from the cpp build dirs (this platform, or every platform in a
release) and `verify` fails the release unless each staged binary is
byte-identical to the one in the release tarball. Makefile: bb-bin,
bb-bin-cross-copy (in release-foundation); the ts release publishes the
platform packages before the meta packages.
bbup installs from npm first (registry tarball of @aztec-foundation/bb-<platform>)
and falls back to the GitHub release assets, which stay published as the
legacy channel.
…able the docs sweep The docs reference checker resolved references: frontmatter paths against the labs submodule root, so the foundation paths added by the docs sweep patch failed the docs build -- only in the merge queue's full build, since fast CI never builds docs. Patch 0005 fixes the checker: paths missing from the labs checkout fall back to the superproject working tree, and standalone checkouts warn instead of failing on paths whose first segment only exists in the parent repo. It is ordered before the re-enabled sweep (now 0006) because it must merge upstream first. labs-docs-refs-check runs the checker in fast CI so a bad reference fails the PR instead of the merge queue.
- publish-npm downloads the bundle with the runner's AWS OIDC credentials (aws s3 cp) instead of the plain-HTTP cache endpoint, extracts it outside the checkout, and publish_npm_release refuses any tarball whose package name is not ours or whose version differs from the release tag. - A re-run no longer tries npm dist-tag add (OIDC trusted publishing only authorizes npm publish); an already-published version is reported, with the manual command when the dist-tag does not point at it. - GCP_PRIVATE_NPM_DEPLOY_KEY: the _KEY suffix keeps ci3/redact treating the service-account JSON as sensitive (the GitHub secret keeps its name), the redact_test expectation is honest again, and ci3's own tests now run in fast CI (make ci3-tests). - NPM_TOKEN is no longer injected into the workflow or forwarded to the build instance. - private-fork-release.yml removed: its API-created tag never triggered ci3.yml, so a fork-dispatched release could never reach publish-npm; the private repo has its own release flow.
…is fully absorbed All five patches (toolchain portal, TEST_CMD_PREFIX, fnd-hashes identity, ipc request-id envelopes, the docs sweep) are upstream byte-identically (matching patch-ids), so the series is now empty. aztec-node also moved noir-projects out of labs/ (labs/noir-projects/labs -> labs/noir-projects): the foundation files that name that layout are updated, and the labs entry in l1-contracts' foundry.lock follows the pin.
# Conflicts: # labs-patches/0006-docs-release-time-syntax-sweep-aztec.js-reference-re.patch
ludamad
changed the base branch from
ad/aztec-foundation-scope
to
ad/ci3-no-runner-ssh-key
August 31, 2026 21:18
…h bb-avm-sim -> bb-ts The barretenberg/ts yarn-install race this ordered around is handled by the chain bb-ts -> bb-avm-sim -> bb-cdb; adding bb-cdb to bb-ts closes that chain into a cycle and make silently drops an edge.
…d base Conflicts with upstream's @aztec-labs scope work and the circuit-recording move are resolved by taking the upstream side and re-applying the scope rename; prettier restores the labs import order.
added 9 commits
August 31, 2026 18:57
… processes On a shared CI machine every concurrent build runs denoise and color_prefix, so the tests' machine-global pgreps matched other jobs' tails, awks and sleeps. denoise gains DENOISE_TMPDIR so the test can watch a private directory; the color_prefix test uses per-run sleep durations and label.
… processes On a shared CI machine every concurrent build runs denoise and color_prefix, so the tests' machine-global pgreps matched other jobs' tails, awks and sleeps. denoise gains DENOISE_TMPDIR so the test can watch a private directory; the color_prefix test uses per-run sleep durations and label.
…nal failures Under CI=1 denoise posts a failing commit status named after the command, so the suite's 'echo failing; exit 42' case stamped a fake failing check onto every commit that ran it.
# Conflicts: # ci3/tests/denoise_test
…cope # Conflicts: # ci3/tests/denoise_test
ludamad
marked this pull request as ready for review
September 1, 2026 00:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #25328. Follow-up to the report that
@aztec/bb.js@6.0.0-nightly.20260826ships abbreporting6.0.0-nightly.20260822.Why it happened — bb.js restores
dest/from a cache keyed on source (+ a release flag), not on the release, so an unchanged tree resurrects an earlier build with the version stamped back then; and the release tarballs were unstripped while bb.js's copies were stripped, so no two channels shipped the same bytes.One path for every distributed binary —
finalize_bb_binaryinbarretenberg/cpp/bootstrap.sh(strip, theninject_version, which re-signs Mach-O), used by:release_tarinbuild_release_dir),build/*/bbat release time,npm packages (plain packable dirs, no workspace/lockfile churn):
@aztec-foundation/bb— meta package with thebbcommand (bin.jsresolves the platform package,BB_BINARY_PATHoverrides), optional deps onbb-linux-x64,bb-linux-arm64,bb-darwin-x64,bb-darwin-arm64,bb-win32-x64;@aztec-foundation/bb-avm— same shape overbb-avm-linux-x64,bb-avm-linux-arm64.barretenberg/ts/scripts/native_packages.sh stage <bb|bb-avm>copies from the cpp build dirs (this platform, or every platform in a release) and finalizes;verifyfails the release unless each staged binary is byte-identical to the tarball's. Makefile:bb-bin,bb-bin-cross-copy(inrelease-foundation); the ts release publishes platform packages before the meta packages.bbup installs from npm first (
https://registry.npmjs.org/@aztec-foundation/bb-<platform>/-/…tgz) and falls back to the GitHub release assets, which remain published as the legacy channel (BBUP_SOURCE=githubforces them). Since the labs toolchain installsbbvia bbup, labs gets the same bytes and abb --versionthat matches the pin.Verified in the devbox (linux/arm64): staged
bbreports exactlyHEAD, is stripped, the shim runs it;release_tar+verifyagree byte-for-byte forbbandbb-avm.Not here: bootstrapping the 9 new packages on npm + trusted publishers (same procedure as the first 23); a labs
set-pinsthat records the npm package version.