Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Documentation/Architecture before changing behavior.
JSONBridges and the swift-json executable. JSONFilterRuntime and JSONCLIEngine are package-private.
JSONHTML and opt-in JSONBridges depend on JSON only; core never re-exports the bridges.
The approved CLI and E2 additions preserve historical product contracts.
JSONNavigation is experimental without a public product until its N1 release gate.
JSONRegexRuntime is package-private. Root package dependencies remain empty.
- Preserve exact bounded decimals, decoded Unicode scalar identity, ordered
objects with order-independent equality, one strict UTF-8 grammar, native
Expand Down
3 changes: 2 additions & 1 deletion Documentation/Architecture/ExtractionManifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,8 @@
"CLI foundation, argv parser, POSIX adapters, prepaid bounded diagnostics and shared-meter package adapters are original Apache-2.0 Swift code. No legacy filter engine or jq source is copied. See Release1.4/Foundation.md for native test-harness reuse provenance.",
"Tests/Fixtures/CLI/NewCases.json is the byte-identical 111-case authored CLI briefing fixture, SHA-256 6f863f3e98d2dea12e59018e7cf3a074ae0ebee5f1ab950a651df72e5eb18a4c. Fixture expected JSON texts use native exact comparison, not a floating-point oracle. The query compiler/VM and lifetime ledger are original Apache-2.0 implementations; no legacy eager evaluator or jq implementation is copied.",
"JSONHTML typed annotations, iterative exact-path binding, static/interactive attachment and coherent JSONDocument span presentation are original Apache-2.0 code. No schema engine, third-party renderer, source archive or new dependency is adapted.",
"Structured forms/cards, finite prefix/preview staging, closed inline scanner, atomic mini-math MathML emitter, canonical PNG/JPEG header admission and line/media-bounded PDF pagination are original Apache-2.0 code. The approved structured concept informed the fixed CSS/JS; no demo Site content or external parser/renderer/codec engine is distributed."
"Structured forms/cards, finite prefix/preview staging, closed inline scanner, atomic mini-math MathML emitter, canonical PNG/JPEG header admission and line/media-bounded PDF pagination are original Apache-2.0 code. The approved structured concept informed the fixed CSS/JS; no demo Site content or external parser/renderer/codec engine is distributed.",
"Experimental N0/N1 JSONNavigation is original Apache-2.0 Swift: iterative traversal and explicit overlap policy reuse unchanged core read-scope/admission/path/meter authorities. No historical navigation/query algorithm or additional dependency is adapted. Three maintainer-provided synthetic S1 JSON fixtures are copied unchanged; their packet and file SHA-256 pins are in Tests/JSONNavigationTests/Fixtures/Provenance.json. Public product promotion remains gated."
],
"policy": "Record actual adaptations and retain applicable upstream copyrights and licenses. Reference inputs and comparison experiments are not distributed runtime dependencies.",
"original_notes": [
Expand Down
106 changes: 106 additions & 0 deletions Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# N1 exact traversal decisions

Scope is N0/N1 only. JSONNavigation depends exclusively on JSON and currently has
no public SwiftPM product. The seven existing products and empty external graph
remain the manifest acceptance contract. Core, schema, Patch, streaming, bridge,
filter/query and renderer behavior is unchanged. The generated CLI source-identity
stamp must refresh when Package.swift/Sources change; this mechanical integration
update adds no CLI behavior, command, language or version change.

## Actual experimental declarations

- `JSONTraversal(document:limits:)` captures one immutable JSONDocument.
- `visit(scope:options:_:)` takes `(JSONTraversalHit, JSON) throws -> JSONTraversalAction`
and returns `JSONOperationReport<JSONTraversalSummary>`.
- `collect(scope:options:maximumRecords:)` uses that same internal execution authority
and returns `JSONOperationReport<JSONTraversalCollection>`.
- `JSONTraversalScope.subtree(at:)` and `.virtual(roots:overlap:)` select absolute
existing typed paths. Virtual defaults to `.rejectAmbiguous`.
- `JSONScopeOverlapPolicy`: rejectAmbiguous, orderedOccurrences, uniquePhysicalNodes.
- `JSONTraversalAction`: continue, skipDescendants, stop.
- `JSONTraversalOptions`: depthLimit, includeScopeRoot, emitKinds, includeSourceSpans.
- `JSONTraversalHit`: absoluteLocation, scopeRootIndex, occurrenceIndex, relativeDepth,
kind and optional coherent sourceSpan. It contains no JSON subtree or cursor.
- `JSONTraversalSummary`: visitedCount, emittedCount, maximumDepthVisited,
scopeRootCount, deduplicatedCount, prunedSubtrees, depthLimitedSubtrees, completion.
- `JSONTraversalCompletion`: complete, restricted, visitorStopped.
- `JSONTraversalFailure`: collectionLimitExceeded(limit:), ambiguousRoots(first:second:).

The [DocC guide](../../../Sources/JSONNavigation/Documentation.docc/Traversal.md)
contains three externally compiled/executed before/after examples. The actual
[consumer](../../../Tools/NavigationQualification/Consumer.swift) uses public JSON
and Navigation declarations, not package meters. Its external-module compiler
probes check core import isolation, hidden meters, non-Sendable cursor rejection
and absence of a fabricated missing kind in both build configurations.

## Exactness and coverage

DFS preorder is the only order offered: object insertion and increasing array
positions. Depth counts edges from each selected root. Full source admission is
mandatory even for a scalar selection, depth zero, filtered-empty selection or
early stop. Root omission/kind filtering changes emission only. Pruning/depth
omission marks coverage restricted, and explicit stop marks visitorStopped. Errors
and cancellation throw. Reports are conservative per-occurrence coverage, not a
union-coverage proof or a global absence query.

All virtual roots resolve before any callback. Reject mode then checks every pair
for equal or ancestor/descendant typed prefixes. Ordered mode preserves root order,
then each subtree's DFS, including repeats. Unique mode remembers **first examined**
typed paths, even excluded/filtered roots, preserving their first depth/origin. It
still explores revisited descendants under the current occurrence's depth policy.
Two equal-valued siblings or scalar-distinct keys never merge. A virtual selection
does not invent a new source root, array index, durable ID or revision fingerprint.

Mapped spans are opt-in core lookups, never synthesized. An unchanged captured old
document continues to describe its old root after another traversal captures a
replacement. No record resolves against a caller-supplied newer root automatically.

## Finite policy and implementation costs

All 49 core metrics/profiles/weights remain unchanged. Existing core read-scope
admission covers root, source bytes/label and source maps. Path records reuse its
pointer, diagnostic and retention policies. The N1 logical overlay reserves 128
bytes for summary/staging, 16 for root-list staging, 64 per planned root or frontier
frame, 64 per ephemeral hit or collected record, and 16 per unique-path reference.
Retained collected paths additionally charge the published 32-byte indexSlot per
component and its key UTF-8 bytes. These are logical weights, not allocation/RSS
measurements; caller-retained callback copies remain caller storage.

Collection has an independent inclusive default of 100,000 records; zero prohibits
emission but allows a genuinely empty result. Negative bounds/depth reject as
configuration errors. Its next-record refusal throws a typed traversal capacity
error and publishes no collection. Source admission uses existing node capacities;
visits do not misleadingly re-charge source nodes, and work stays cumulative across
roots, pair scans, dedup, emitted paths, source-span lookups, output and cleanup.

Initial reject/unique modes use exact metered linear comparisons. Pair validation
and unique-node scans can be quadratic. Core path snapshots along a deep frontier
can also use quadratic path metadata. The prototype intentionally makes no
linear-total-time, indexed-lookup, zero-copy or speed claim. Every owned comparison
loop is cancellable/budgeted, so exhaustion is an operational error, never a partial
success. A future compact representation/index needs its own measured review.

The invocation and its synchronous scope never escape into a callback. JSONTraversal,
metadata/options and collected values are checked Sendable; each simultaneous call
creates its own scope. A package-only nil-by-default checkpoint observer supports
owned-loop cancellation qualification; no public consumer can install it.

## Qualification and promotion boundary

The reviewed new inventory is 37 nonparameterized Navigation tests, including the
three pinned synthetic packet documents. Existing function identities and all 325
native/Linux parameter invocations remain; WASI retains its separate 288. Full roots
become 583 native, 567 Linux and 479 WASI, and shared-state counts 42/41. All 126
maintenance identities, corpus policies and 49 cap probes remain required.

`Scripts/ci.py --lane navigation` checks exact focused discovery, Debug/Release,
explicit strict-memory compilation and external compiler/consumer/DocC probes.
Native/Linux run the new consumer and full existing suites; safety includes Navigation
ASan plus its shared-root TSan function. WASI uses the updated exact root inventory.
Current commands/receipts, failures and platform gaps accompany the return; this
decision document does not assert a pending platform run passed. Full release-gate
qualification and explicit product review precede promotion to an installable product.

BFS, scalar-sorted order, prefix snapshots, JSONInspection, classifiers, snapshots,
semantic/durable digests, projections, editing, retrieval, actors, rendering and CLI
verbs are explicitly deferred. No EngineBench implementation or performance claim.
37 changes: 37 additions & 0 deletions Documentation/Architecture/StructuralIntelligence/Progress.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{
"schema_version": 1,
"research_base": "507dbf9ad4b545a5d60d7d6acf6b814795ec3099",
"implementation_base": "037e735dc6224789e4d04d2c18d15b8d27440198",
"packet_sha256": "d758f75d84271a97da515bf7b7ef70de7a61e91fdf949a511fe0fca2708156f8",
"N0": "source authorities reconciled; core-only seven-witness consumer and package-scoped one-ledger composition compiled/executed",
"N1": "experimental physical/virtual DFS, bounded collection and 37 behavior functions implemented; functional native/Linux/WASI proofs passed at 6e5e5fa; public product promotion remains gated",
"public_product": false,
"public_products_preserved": 7,
"core_semantic_changes": false,
"deferred": [
"BFS",
"N2",
"N3",
"P1",
"P2",
"R1",
"R2",
"actors",
"persistence",
"query grammar",
"rendering",
"CLI verbs",
"EngineBench"
],
"qualification": {
"evidence_source": "6e5e5fa7d8c4c7e8ce1c4fe14bb3323f1ddc7ac8",
"native_run": 38781500802,
"native_boundary": "native/safety/docs/archive qualification commands passed; native/archive artifact upload failed on raw DocC colon filenames; preserved as a failed workflow",
"linux_run": 38781500823,
"linux_boundary": "both architectures qualification and safety passed",
"wasi_boundary": "77-command full Node/limited WasmKit lane passed with unchanged source; supplementary 37-function Navigation Debug/Release runs passed in WasmKit",
"mobile_boundary": "iPhoneOS/iPhoneSimulator 27.2 target compilation passed, no device/runtime claim",
"transport_successor": "native catalogue is packaged byte-for-byte into a verified tar/manifest before upload; fresh hosted outcomes and materialized evidence are supplied by PR 34 and the return packet",
"product_review": "experimental target only; no public product or merge/release action"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# N0 source authority

The research baseline is `507dbf9ad4b545a5d60d7d6acf6b814795ec3099`.
The reconciled implementation base is main
`037e735dc6224789e4d04d2c18d15b8d27440198`, tree
`5c53288c50ae8a1999d889f0a8c4f0cb14dca913`. Origin was fetched before edits.
PR #33 forward-ports maintenance repairs after the research snapshot; no reset,
old scaffold or historical navigation implementation is used.

At reconnaissance all nine main checks passed: Native CI 38735450692 and Linux
CI 38735450703. Those are baseline results, not qualification of this increment.
The latest published release observed was 1.5.0; main's `1.6.0-dev` CLI metadata
does not establish a publication. Draft #31 owns opt-in value/schema DSLs, and a
separate schema-example worktree has active edits. Neither is imported or modified.
The primary checkout stays on main; implementation uses
`codex/structural-navigation-n0-n1` in an isolated existing feature worktree.

## Semantic delegates and owners

Paths below identify code ownership, not a second resolver API. All new owned
phases use the **same** read-scope state and its noncopyable core-domain meter.

| New operation or fact | Existing authority / owner | Budget boundary |
| --- | --- | --- |
| Captured canonical source | [JSONDocument](../../../Sources/JSON/Document/JSONDocument.swift) and JSON | Constructor captures trusted immutable values; each invocation admits its full root/source/map |
| Admission and source lifetime | [JSONReadScope.init/check/complete/invalidate](../../../Sources/JSON/Reading/JSONReadScope.swift) | Exactly one root/source admission and terminal ledger; complete releases scope ownership |
| Exact keys / insertion order | [JSONObject](../../../Sources/JSON/Value/JSONObject.swift), JSONKey | Visit ordinal members; no String sort, dictionary conversion or duplicate index |
| Typed physical selection | JSONReadScope.select inside run | Strict key/index parent checks and existing JSONAccessFailure; no Pointer-token reinterpretation |
| Requested scope-key admission | JSONAdmission.text(key: true), as in JSONReadCursor's JSONKey subscript | Caller query keys consume the same cumulative key/text capacities before path staging or lookup |
| Typed path staging | JSONReadScope.append and JSONReadPath | Existing pointerTokens/pointerBytes/diagnosticBytes checks, path copy/UTF-8 work and live refunds |
| Meter/operational failure | [JSONWorkMeter](../../../Sources/JSON/Limits/JSONWorkMeter.swift), JSONCodingState | Shared cumulative work, capacity/retention, cancellation and terminal health; never reset |
| DFS child enumeration | JSON array indices / JSONObject ordinal collection access | New iterative frontier; each node/edge/frame/callback handoff checked and charged |
| Overlap / unique-path comparison | [JSONComparison.textEquals](../../../Sources/JSON/Value/JSONComparison.swift) + typed integer components | New metered component/prefix loops; no Set<String>, unmetered path hash, or value-based dedup |
| Decimal and text values given to visitor | JSON/JSONNumber/JSONKey | Original source value; no conversion or re-admission per hit |
| Pointer serialization / differential oracle | [JSONPointer](../../../Sources/JSON/Pointer/JSONPointer.swift) | Explicit separate caller operation; walker never calls a public helper to start a fresh owned quota |
| Native projection / presence oracle | [JSONReadCursor](../../../Sources/JSON/Reading/JSONReadCursor.swift), JSONPresence | Compiler/behavior witnesses only; no cursor is a traversal result or task argument |
| Optional value spans | [JSONSourceMap.span](../../../Sources/JSON/Document/JSONSourceMap.swift) | Same meter, only coherent captured maps; replacingRoot supplies no map |
| Successful resource reporting | JSONOperationReport / JSONOperationStatistics | Existing accountingVersion/domain/counts/peak convention; no successful report after error |
| Mutations | JSONPatch / JSONMergePatch | Unchanged and outside N1; traversal has no edits or writable value view |

`JSONReadScope` and `JSONWorkMeter` are already `package`, so no core SPI,
public meter protocol or package helper change is needed. Actual target compilation
and the inclusive work-budget tests prove this access/composition boundary. The
public N0 consumer compiled and ran seven core-only witnesses before N1: root versus
empty key, typed key/index, exact Unicode keys, presence, native projection and
invalid array-index rejection. Compiler receipts retain the actual source/toolchain.

## Collision and dialect reconciliation

| Historical/proposed surface | Disposition |
| --- | --- |
| JSONCursor / JSONNavigationCursor | Rejected: guarded core read cursor already owns synchronous one-node access |
| JSONNodePath / dotted flattening | Rejected: canonical components are JSONLocationComponent; root [] and empty key differ |
| New JSON. facade / JSONPath grammar | Rejected: neither necessary nor an RFC 9535 implementation |
| JSONFilterRuntime paths/getpath/walk | Existing private query dialect can accept negative indices and language-level missing behavior; not a strict typed scope delegate |
| JSONHTML exact paths and previews | Existing rendering authority; no renderer import/edit or duplicate preview logic |
| JSONTraversal and metadata names | New target-qualified names; no existing core/sibling definition found at the recorded base |
| Sketch BFS / inspection / snapshots / projections | Deferred; no reserved BFS label, parallel model, digest or classifier added |

The packet's declarations were illustrative. N1 returns existing
JSONOperationReport envelopes, passes the source value separately to a synchronous
visitor, and keeps records metadata-only. Its module is experimental; **no public
SwiftPM product is added**. API shape and all remaining release gates are recorded
in [API-DECISIONS](API-DECISIONS.md).
12 changes: 12 additions & 0 deletions Documentation/CI/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Native CI

## Experimental N1 branch

The Structural Intelligence branch adds exactly 37 Navigation functions and one
shared-root safety function, keeping every existing identity/case. Root expectations
are 583 native/567 Linux with 325 parameter invocations; TSan is 42/41. WASI's
separate core scope is 479 functions/288 parameters. No new public product is exposed.
`--lane navigation` checks focused discovery, Debug/Release, explicit strict-memory
compilation and the external compiler/consumer/experimental DocC audit. Native/Linux
include that consumer alongside their existing gates; all 49 core cap outcomes and
126 restored maintenance identities remain required. The following earlier counts
retain their original source context; actual source-bound receipts establish passes.

This main-based forward-port retains seven products and restores the entire
1.2.1 regression inventory. Current native root discovery is 546 functions and
325 parameter invocations; Linux is 530/325. Full ASan uses the same roots;
Expand Down
18 changes: 12 additions & 6 deletions Documentation/Qualification/TestDiscovery.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
"JSONIntegrationTests": 20,
"JSONHTMLTests": 97,
"JSONBridgesTests": 10,
"JSONCLITests": 88
"JSONCLITests": 88,
"JSONNavigationTests": 37
},
"parameterized_invocations": 325,
"qa_functions": {
Expand All @@ -28,7 +29,8 @@
"JSONIntegrationTests": 10,
"JSONHTMLTests": 5,
"JSONBridgesTests": 1,
"JSONCLITests": 1
"JSONCLITests": 1,
"JSONNavigationTests": 1
},
"coverage_unmapped": {
"Sources/JSON/Pointer/JSONLocationComponent.swift": "declaration-only enum",
Expand All @@ -52,7 +54,8 @@
"JSONBridges",
"JSONFilterRuntime",
"JSONCLIEngine",
"JSONCLI"
"JSONCLI",
"JSONNavigation"
],
"portable_root_functions": {
"JSONTests": 190,
Expand All @@ -63,7 +66,8 @@
"JSONIntegrationTests": 20,
"JSONHTMLTests": 81,
"JSONBridgesTests": 10,
"JSONCLITests": 88
"JSONCLITests": 88,
"JSONNavigationTests": 37
},
"wasi_root_functions": {
"JSONTests": 190,
Expand All @@ -73,7 +77,8 @@
"JSONPatchTests": 16,
"JSONIntegrationTests": 20,
"JSONHTMLTests": 81,
"JSONBridgesTests": 10
"JSONBridgesTests": 10,
"JSONNavigationTests": 37
},
"wasi_parameterized_invocations": 288,
"portable_shared_state_functions": {
Expand All @@ -84,7 +89,8 @@
"JSONIntegrationTests": 10,
"JSONHTMLTests": 4,
"JSONBridgesTests": 1,
"JSONCLITests": 1
"JSONCLITests": 1,
"JSONNavigationTests": 1
},
"export_functions": {
"JSONSchemaTests": 1,
Expand Down
Loading