Skip to content

perf(e2e): batch same-sender setup txs into single txs#24584

Closed
spalladino wants to merge 1 commit into
spl/e2e-overlap-setup-txsfrom
spl/e2e-batch-setup-txs
Closed

perf(e2e): batch same-sender setup txs into single txs#24584
spalladino wants to merge 1 commit into
spl/e2e-overlap-setup-txsfrom
spl/e2e-batch-setup-txs

Conversation

@spalladino

@spalladino spalladino commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Context

#24569 overlapped the fees harness's independent setup txs with Promise.all, but the fix-phase capture showed only ~1/3 of the projected gain (−162s). The overlap was real (tx:mint busy/total ≈ 0.68), yet concurrent txs still land in consecutive 12s slots: the PXE serializes simulateTx/proveTx on a single queue, so the second concurrent tx is only proven after the first and misses the first's slot. Overlapping cannot remove whole slots — only batching can.

This PR batches same-sender setup calls into a single BatchCall tx: one proof, one tx, one slot.

What was batched

FeesTest.applyFundAliceWithBananas (end-to-end/src/single-node/fees/fees_test.ts) previously sent alice's private and public banana mints as two concurrent txs. Both are alice sends on BananaCoin touching disjoint balances, so they now ride one BatchCall([mint_to_private, mint_to_public]) tx. The private-balance before/after assertion is preserved as the behavioral check; the public mint's effect is checked by the downstream tests that spend alice's public bananas. Precedent for the pattern already lives in this tree (mintNotes in fixtures/token_utils.ts batches multiple mint_to_private calls).

This touches the fees suites that fund alice in setup: account_init, gas_estimation, private_payments, and fee_juice_payments.

Surveyed and skipped

  • applyFPCSetup: a single deploy:fpc tx — no same-sender pair inside the phase. Folding the FPC deploy into the mint batch (one class publication + app calls is within the class-log limit) would remove the remaining consecutive slot, but it merges two harness phases that four test files compose differently via Promise.all, and deploy-inside-BatchCall is not a proven pattern here — a restructure, not a mechanical merge. Left as a possible follow-up.
  • applySponsoredFPCSetup: no on-chain tx — it only calls wallet.registerContract locally; the sponsored FPC is genesis-funded and its class preloaded. Nothing to batch.
  • applyBaseSetup chain: the auth-registry publish short-circuits (genesis-seeded by perf(e2e): seed standard contracts at genesis #24568) and applyDeployBananaToken is a single deploy tx. No consecutive same-sender pair.
  • Two different contract deploys cannot share a tx (MAX_CONTRACT_CLASS_LOGS_PER_TX = 1), which rules out any two-deploy batch (e.g. cross-chain token+bridge).
  • Cross-chain harness deploys and bot/src/factory.ts are out of scope (the latter lives on perf(bot): parallelize independent bot factory setup steps #24581's stack).

Local span evidence (account_init, 5/5 green on all runs)

Suite-scoped beforeAll spans from TEST_TIMING_FILE captures:

run tx:mint count tx:mint busyMs deploy:fpc busyMs beforeHooksMs
baseline (before) 2 23618 23713 71607
run 1 (after) 1 11834 24065 72372
run 2 (after) 1 11807 23743 71678

The mint phase drops from two consecutive slots (~23.6s) to one (~11.8s), one proof and one tx fewer per shard. account_init's total beforeAll is flat locally because its mint phase runs concurrently with applyFPCSetup and deploy:fpc still pays 2 slots (PXE proves the mint batch first), remaining the phase's critical path — the direct wall-clock win lands on suites where the mints are the phase (e.g. fee_juice_payments), and the fewer-proofs contention win applies everywhere.

Stacking

Stacked on #24569 (spl/e2e-overlap-setup-txs), itself stacked on #24564. Retarget to merge-train/spartan-v5 once #24569 merges.

Fixes A-1409

Measured impact

The batch lands exactly as the local runs predicted: 13 mint txs merged run-wide, with the wall-clock
win concentrated on the suites where the mint phase is the critical path.

  • tx:mint: 42 → 29 occurrences (one merged per fees shard); busyMs 523s → 431s (−92s of tx/proving
    work). In the PR run busyMs equals totalMs — there are no overlapping mint txs left to union.
  • Suites where the mint is critical-path: failures −12.0s, fee_juice_payments −11.5s (≈ one 12s
    slot each, as designed).
  • Suites where the mint overlaps applyFPCSetup are flat as expected: account_init −0.4s,
    private_payments.parallel +2s (×8 shards), gas_estimation.parallel +3s (×3) — deploy:fpc is
    unchanged (13 occurrences, ~23s / 2 slots each) and remains their setup critical path, so removing
    a slot from the mint branch of the Promise.all doesn't move the union.
  • Run-wide sum over common suites: −8s, i.e. neutral outside the two critical-path suites; the noise
    envelope on untouched suites in this run pair is ±20–37s (bot −20s, private_initialization +37s).

Net: ~−23s wall on two suites plus 13 fewer txs/proofs per run (less proving contention and one less
tx per fees shard through the sequencer). The remaining ~12s/shard lever on the overlapped fees
suites is folding the FPC deploy into the same batch (one class publication + app calls is within
MAX_CONTRACT_CLASS_LOGS_PER_TX), noted above as a follow-up.

Baseline CI run 1783394912998666 (#24569 ci/x-fast, the stack base). PR CI run 1783432285938536
(ci/x-fast).

@spalladino spalladino added ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure wip Work in progress and removed wip Work in progress labels Jul 7, 2026
@spalladino

Copy link
Copy Markdown
Contributor Author

Folded into #24569 per review: the two PRs touched the same mint path in fees_test.ts (this one rewrote the Promise.all overlap that #24569 introduced into a single BatchCall), so they are now a single PR. The commit here (7b9cd12) was fast-forwarded onto spl/e2e-overlap-setup-txs unchanged, and #24569's description now covers both changes, including this PR's measured impact. Fixes A-1409 moves there too.

@spalladino spalladino closed this Jul 7, 2026
spalladino added a commit that referenced this pull request Jul 8, 2026
Stacked on #24564; retarget to `merge-train/spartan-v5` after it merges.
(Unifies former #24584 into this PR — the two touched the same mint path
in `fees_test.ts`, so they now land together.)

Restructures the serial setup-tx chains in the fees and cross-chain e2e
harnesses so independent setup transactions stop paying one production
slot each: independent txs run concurrently, and same-sender calls are
batched into a single tx (one proof, one tx, one slot). Part of the
round-4/5 e2e speedup effort.

## Commits

- **perf(e2e): overlap fees harness setup txs** — after `FeesTest.setup`
deploys BananaCoin, the BananaFPC deploy, the SponsoredFPC registration
(a PXE registration, no L2 tx), and Alice's banana mints each depend
only on the token, so the fees suites (`account_init`, `gas_estimation`,
`private_payments`) now run them under a `Promise.all`. The cross-chain
harness (`deployAndInitializeTokenAndBridgeContracts`) deploys the L2
token and its bridge concurrently: the bridge takes the token address as
a constructor arg, but that address is deterministic before the deploy
tx mines (the deploy is now constructed with an explicit `deployer`,
same address derivation as the previous send-time lock), and the bridge
constructor only stores it without calling into the token.
- **perf(e2e): parallelize dual token mints** + **perf(e2e): batch
same-sender setup txs into single txs** — `applyFundAliceWithBananas`
sent Alice's private and public banana mints back-to-back (measured 100%
serial: `tx:mint` busyMs == totalMs on the base). The first commit made
them concurrent; the second replaces the concurrency with a single
`BatchCall([mint_to_private, mint_to_public])` tx, because measurement
showed concurrent same-sender txs still pay *consecutive* slots: the PXE
serializes `simulateTx`/`proveTx` on a single queue, so the second tx is
only proven after the first and misses the first's slot. Overlapping
cannot remove whole slots — only batching can. Both mints are Alice
sends on `BananaCoin` touching disjoint balances; the private-balance
before/after assertion is preserved, and the public mint's effect is
checked by the downstream tests that spend Alice's public bananas.
Precedent for the pattern already lives in this tree (`mintNotes` in
`fixtures/token_utils.ts`).

## Dependency analysis

- FPC deploy needs the token address (constructor arg) → runs after the
token deploy, concurrent with the mint batch.
- Mints need the token → concurrent with the FPC deploy, and the
private/public mints are mutually independent (disjoint balances), so
they share one tx.
- SponsoredFPC setup is a wallet registration only → free to join the
`Promise.all`.
- TokenBridge's initializer stores `{token, portal}` in its own storage
and never calls the token, so the bridge deploy does not need the token
to be mined first.
- #24564 invariants preserved: `BANANA_COIN_SALT`/`BANANA_FPC_SALT` +
`deployer: alice` are untouched, and the genesis-funded-address
assertion in `applyFPCSetup` still runs (and passed locally).

## Same-sender concurrency

Concurrent `.send()` calls from the same sender through the same
TestWallet are safe: the PXE serializes `simulateTx`/`proveTx` through a
`SerialQueue` (`#putInJobQueue`), so simulations never interleave and
only the block-inclusion waits overlap. The setup txs pay fees from
Alice's public fee-juice balance (`PREEXISTING_FEE_JUICE`) with random
tx nonces, so there are no nullifier or note conflicts between the
concurrent txs. Verified locally with full `Promise.all` of the
send+wait calls; no NO_WAIT fallback or cross-sender split was needed.

## Surveyed and skipped

- Batching token class+instance publication: not applicable on this base
— `DeployMethod.request()` already merges class publication, instance
publication, and the constructor call into a single execution payload
sent as one tx. Measured locally, `deploy:token` is a single ~12s (1
slot) occurrence; the ~24s occurrences in CI span data are inclusion
latency at CI cadence, not a second tx.
- `applyFPCSetup`: a single `deploy:fpc` tx — no same-sender pair inside
the phase. Folding the FPC deploy into the mint batch (one class
publication + app calls is within the class-log limit) would remove the
remaining consecutive slot, but it merges two harness phases that four
test files compose differently via `Promise.all`, and
deploy-inside-BatchCall is not a proven pattern here — a restructure,
not a mechanical merge. Left as a possible follow-up.
- `applySponsoredFPCSetup`: no on-chain tx — it only calls
`wallet.registerContract` locally; the sponsored FPC is genesis-funded
and its class preloaded. Nothing to batch.
- Two different contract deploys cannot share a tx
(`MAX_CONTRACT_CLASS_LOGS_PER_TX = 1`), which rules out any two-deploy
batch (e.g. cross-chain token+bridge) — those stay concurrent rather
than batched.

## Local verification (12s slots; CI runs at its own cadence)

Overlap commits (fees + cross-chain):

- Base (`account_init`, spans on): 5/5 passed; beforeAll 83.5s =
`setup:auth-registry` 32.0s + `deploy:token` 12.0s + `tx:mint` 23.9s
(busyMs == totalMs, fully serial) + `deploy:fpc` 11.8s.
- After overlap: 5/5 passed; beforeAll 72.2s (−11.3s); `deploy:fpc`
fully overlapped with the mint window.
- `gas_estimation`: 3/3 passed; `private_payments`: 8/8 passed;
`token_bridge` (cross-chain overlap): 7/7 passed — `deploy:token` 12.1s
and `deploy:bridge` 23.7s ran concurrently (union ≈ 23.7s vs ~48s serial
on the CI baseline); `l2_to_l1`: 6/6 passed (regression check on the
untouched path).

Batch commit (`account_init`, 5/5 green on all runs):

| run | tx:mint count | tx:mint busyMs | deploy:fpc busyMs |
beforeHooksMs |

|-----|---------------|----------------|-------------------|---------------|
| before batch | 2 | 23618 | 23713 | 71607 |
| run 1 (after) | 1 | 11834 | 24065 | 72372 |
| run 2 (after) | 1 | 11807 | 23743 | 71678 |

The mint phase drops from two consecutive slots (~23.6s) to one
(~11.8s), one proof and one tx fewer per shard. No intermittent failures
across any run.

## Measured impact

Measured on CI in two steps (each commit group had its own PR run before
unification).

**Overlap** (PR run 1783394912998666 vs #24564's run 1783375878950264):
−162s summed across shards. Per-suite beforeHooks:
private_payments.parallel −16.0s (×8), gas_estimation.parallel −15.5s
(×3), token_bridge −12.0s, fee_juice_payments −12.0s, account_init
−11.5s. The overlap is real but partial: `tx:mint` busyMs/totalMs = 0.68
run-wide — concurrent txs land in consecutive slots because the PXE
serializes proving (the finding that motivated the batch commit).
Cross-chain `token_bridge` collapses its deploy phase from serial 35.9s
to concurrent ~23.4s (union ≈ max, as designed).

**Batch** (PR run 1783432285938536 vs 1783394912998666): `tx:mint` 42 →
29 occurrences (one merged per fees shard); busyMs 523s → 431s (−92s of
tx/proving work; busyMs == totalMs in the PR run — no overlapping mint
txs left to union). Wall-clock lands where the mint is the setup
critical path: `failures` −12.0s, `fee_juice_payments` −11.5s (≈ one 12s
slot each). Suites where the mint overlaps `applyFPCSetup` are flat
(`account_init` −0.4s, `private_payments.parallel` +2s,
`gas_estimation.parallel` +3s): `deploy:fpc` is unchanged (13
occurrences, ~2 slots each) and remains their critical path, so removing
a slot from the mint branch of the `Promise.all` doesn't move the union.
Run-wide sum over common suites: −8s; noise envelope on untouched suites
±20–37s.

Net vs #24564's baseline: roughly −185s summed across shards, plus 13
fewer txs/proofs per run (less proving contention through the
sequencer). The remaining ~12s/shard lever on the overlapped fees suites
is folding the FPC deploy into the same batch, noted above as a
follow-up.

Fixes A-1406
Fixes A-1409
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure wip Work in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant