Skip to content

zero-swift: deep JSON without recursion; nesting limit 1,500 - #17

Merged
tmm merged 2 commits into
mainfrom
swift-json-depth
Oct 22, 2026
Merged

tmm merged 2 commits into
mainfrom
swift-json-depth

Conversation

@tmm

@tmm tmm commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Option (c), approved by tmm: iterative JSON code plus a lower nesting limit, with the limit documented as a difference from JS.

What changed:

  • JSONParser is one loop with an explicit stack. Key order, duplicate-key handling, and error messages and offsets are unchanged.
  • JSONSerializer.write, JSONValue.hash, QueryHash value hashing and JSONValue.== don't recurse. == recurses for the first 64 levels and switches to an explicit stack below that.
  • JSONParser.maxDepth drops from 5,000 to 1,500. Freeing a value still recurses, in Array's and JSONObject's storage. At 1,500 levels that fits a 512 KB stack in a release build.
    • Measured on a 512 KB thread: a release build frees arrays and objects to at least 3,000 levels. A debug build frees arrays to 2,000 levels, but only objects to about 1,000.
  • JSParity.md documents the limit. JS has no limit short of its own stack, and zero-cache writes about 4,100 levels. A row whose JSON column is nested deeper than about 1,495 levels makes the client reject the poke, disconnect, and hit the same row after reconnecting. Real data, such as rich text, rarely passes a few hundred levels.

Tests:

  • JSONDepthTests checks values exactly maxDepth deep on a 512 KB thread. On the old code it crashes with signal 11.
  • SweepWireTests.deepRowValue checks a poke at the limit on 512 KB, including freeing it, and checks that one more level is rejected.

Cost (release, same process, old vs new):

  • Parsing a poke: +7–9%.
  • Parsing payloads with many nested containers: +12–22%.
  • == on large nested values: +35–40%.
  • Client benchmarks: no change outside noise.

Tested: 209 tests pass under ulimit -s 512. Also lint, ClientE2ETests (2) and the zbugs-swift build.

Base automatically changed from swift-json-keys to main October 22, 2026 06:24
…cursion

JSONParser accepts 5,000 levels, but parsing recursed and overflowed a
512 KB stack (a macOS test or cooperative thread) at about 300 levels in
a debug build. The parser, JSONSerializer, the query hash's visitValue,
and JSONValue's hash now keep the arrays and objects they are in on an
explicit stack. JSONValue's == recurses for the first 64 levels, which is
faster, and goes on without recursion past them.

Freeing a deep value still recurses (Array's and JSONObject's deinit):
a 5,000-deep value fits in 1 MB in a release build, not in 512 KB.

SweepWireTests no longer needs an 8 MB stack to parse; deepRowValue runs
on 512 KB and frees on 8 MB. JSONDepthTests parses, writes, compares, and
hashes 5,000-deep arrays and objects on a 512 KB thread.

Amp-Thread-ID: https://ampcode.com/threads/T-c513a037-a257-45e0-b9a9-15c15edfe859
Co-authored-by: tmm <tmm@tmm.dev>
@tmm
tmm force-pushed the swift-json-depth branch from 79afdb7 to 03677a9 Compare October 22, 2026 06:36
… small stack

Freeing a value still recurses in Array's and JSONObject's storage. At
1,500 levels it fits a 512 KB stack in a release build. Documents the
limit as a difference from JS in JSParity.md.

Amp-Thread-ID: https://ampcode.com/threads/T-c513a037-a257-45e0-b9a9-15c15edfe859
Co-authored-by: tmm <tmm@tmm.dev>
@tmm tmm changed the title zero-swift: parse, write, compare, and hash deep JSON without recursion zero-swift: deep JSON without recursion; nesting limit 1,500 Oct 22, 2026
@tmm
tmm marked this pull request as ready for review October 22, 2026 14:53
@tmm
tmm merged commit 151ede9 into main Oct 22, 2026
9 checks passed
@tmm
tmm deleted the swift-json-depth branch October 22, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants