From f83a638114dfd3e684e92e35818d5b2c8d0345cd Mon Sep 17 00:00:00 2001 From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com> Date: Thu, 15 Oct 2026 23:56:32 +0200 Subject: [PATCH 1/6] Integrate exact Navigation traversal over composition-enabled main Replay original 94369e1 from PR #34; retain composition source, lanes, fixtures and accounting. Conflicts resolved additively; CLI identity regenerated. --- AGENTS.md | 1 + .../Architecture/ExtractionManifest.json | 3 +- .../StructuralIntelligence/API-DECISIONS.md | 106 ++++++++ .../StructuralIntelligence/Progress.json | 12 + .../SOURCE-AUTHORITY.md | 63 +++++ Documentation/CI/README.md | 12 + .../Qualification/TestDiscovery.json | 18 +- Documentation/README.md | 2 + Package.swift | 3 + README.md | 5 + Scripts/README.md | 1 + Scripts/ci.py | 19 +- Scripts/qualify_navigation.py | 116 +++++++++ Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +- .../Documentation.docc/JSONNavigation.md | 32 +++ .../Documentation.docc/Traversal.md | 76 ++++++ Sources/JSONNavigation/JSONTraversal.swift | 229 +++++++++++++++++ .../JSONNavigation/JSONTraversalTypes.swift | 152 +++++++++++ .../Fixtures/Provenance.json | 19 ++ Tests/JSONNavigationTests/Fixtures/paths.json | 15 ++ .../Fixtures/source-revisions.json | 18 ++ .../Fixtures/structures.json | 56 ++++ .../NavigationBoundaryTests.swift | 240 ++++++++++++++++++ .../PacketWitnessTests.swift | 39 +++ .../PhysicalTraversalTests.swift | 68 +++++ .../VirtualScopeTests.swift | 99 ++++++++ Tools/NavigationQualification/Consumer.swift | 80 ++++++ 27 files changed, 1476 insertions(+), 10 deletions(-) create mode 100644 Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md create mode 100644 Documentation/Architecture/StructuralIntelligence/Progress.json create mode 100644 Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md create mode 100644 Scripts/qualify_navigation.py create mode 100644 Sources/JSONNavigation/Documentation.docc/JSONNavigation.md create mode 100644 Sources/JSONNavigation/Documentation.docc/Traversal.md create mode 100644 Sources/JSONNavigation/JSONTraversal.swift create mode 100644 Sources/JSONNavigation/JSONTraversalTypes.swift create mode 100644 Tests/JSONNavigationTests/Fixtures/Provenance.json create mode 100644 Tests/JSONNavigationTests/Fixtures/paths.json create mode 100644 Tests/JSONNavigationTests/Fixtures/source-revisions.json create mode 100644 Tests/JSONNavigationTests/Fixtures/structures.json create mode 100644 Tests/JSONNavigationTests/NavigationBoundaryTests.swift create mode 100644 Tests/JSONNavigationTests/PacketWitnessTests.swift create mode 100644 Tests/JSONNavigationTests/PhysicalTraversalTests.swift create mode 100644 Tests/JSONNavigationTests/VirtualScopeTests.swift create mode 100644 Tools/NavigationQualification/Consumer.swift diff --git a/AGENTS.md b/AGENTS.md index 63617e4..e5e6be1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,6 +8,7 @@ Documentation/Architecture before changing behavior. JSONBridges and the swift-json executable. JSONFilterRuntime and JSONCLIEngine are package-private. JSONHTML and opt-in JSONBridges depend on JSON only; core never re-exports the bridges. The approved CLI and E2 additions preserve historical product contracts. + JSONNavigation is experimental without a public product until its N1 release gate. JSONRegexRuntime is package-private. Root package dependencies remain empty. - Preserve exact bounded decimals, decoded Unicode scalar identity, ordered objects with order-independent equality, one strict UTF-8 grammar, native diff --git a/Documentation/Architecture/ExtractionManifest.json b/Documentation/Architecture/ExtractionManifest.json index 0dd8127..14aa49e 100644 --- a/Documentation/Architecture/ExtractionManifest.json +++ b/Documentation/Architecture/ExtractionManifest.json @@ -261,7 +261,8 @@ "Tests/Fixtures/CLI/NewCases.json is the byte-identical 111-case authored CLI briefing fixture, SHA-256 6f863f3e98d2dea12e59018e7cf3a074ae0ebee5f1ab950a651df72e5eb18a4c. Fixture expected JSON texts use native exact comparison, not a floating-point oracle. The query compiler/VM and lifetime ledger are original Apache-2.0 implementations; no legacy eager evaluator or jq implementation is copied.", "JSONHTML typed annotations, iterative exact-path binding, static/interactive attachment and coherent JSONDocument span presentation are original Apache-2.0 code. No schema engine, third-party renderer, source archive or new dependency is adapted.", "Structured forms/cards, finite prefix/preview staging, closed inline scanner, atomic mini-math MathML emitter, canonical PNG/JPEG header admission and line/media-bounded PDF pagination are original Apache-2.0 code. The approved structured concept informed the fixed CSS/JS; no demo Site content or external parser/renderer/codec engine is distributed.", - "Experimental synchronous composition facade and its monotone external reservations are original Apache-2.0 implementations reusing native meter, admission, codec, Pointer and URI helpers. Contract tests concretize a maintainer-provided research scaffold; private packet evidence is excluded from source distribution. No third-party semantic engine or dependency is adapted. Compiler substitution fixtures materialize maintainer-provided R1 public-interface review cases, including exact work, retention and caught-failure restoration probes." + "Experimental synchronous composition facade and its monotone external reservations are original Apache-2.0 implementations reusing native meter, admission, codec, Pointer and URI helpers. Contract tests concretize a maintainer-provided research scaffold; private packet evidence is excluded from source distribution. No third-party semantic engine or dependency is adapted. Compiler substitution fixtures materialize maintainer-provided R1 public-interface review cases, including exact work, retention and caught-failure restoration probes.", + "Experimental N0/N1 JSONNavigation is original Apache-2.0 Swift: iterative traversal and explicit overlap policy reuse unchanged core read-scope/admission/path/meter authorities. No historical navigation/query algorithm or additional dependency is adapted. Three maintainer-provided synthetic S1 JSON fixtures are copied unchanged; their packet and file SHA-256 pins are in Tests/JSONNavigationTests/Fixtures/Provenance.json. Public product promotion remains gated." ], "policy": "Record actual adaptations and retain applicable upstream copyrights and licenses. Reference inputs and comparison experiments are not distributed runtime dependencies.", "original_notes": [ diff --git a/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md b/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md new file mode 100644 index 0000000..e45407f --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md @@ -0,0 +1,106 @@ +# N1 exact traversal decisions + +Scope is N0/N1 only. JSONNavigation depends exclusively on JSON and currently has +no public SwiftPM product. The seven existing products and empty external graph +remain the manifest acceptance contract. Core, schema, Patch, streaming, bridge, +filter/query and renderer behavior is unchanged. The generated CLI source-identity +stamp must refresh when Package.swift/Sources change; this mechanical integration +update adds no CLI behavior, command, language or version change. + +## Actual experimental declarations + +- `JSONTraversal(document:limits:)` captures one immutable JSONDocument. +- `visit(scope:options:_:)` takes `(JSONTraversalHit, JSON) throws -> JSONTraversalAction` + and returns `JSONOperationReport`. +- `collect(scope:options:maximumRecords:)` uses that same internal execution authority + and returns `JSONOperationReport`. +- `JSONTraversalScope.subtree(at:)` and `.virtual(roots:overlap:)` select absolute + existing typed paths. Virtual defaults to `.rejectAmbiguous`. +- `JSONScopeOverlapPolicy`: rejectAmbiguous, orderedOccurrences, uniquePhysicalNodes. +- `JSONTraversalAction`: continue, skipDescendants, stop. +- `JSONTraversalOptions`: depthLimit, includeScopeRoot, emitKinds, includeSourceSpans. +- `JSONTraversalHit`: absoluteLocation, scopeRootIndex, occurrenceIndex, relativeDepth, + kind and optional coherent sourceSpan. It contains no JSON subtree or cursor. +- `JSONTraversalSummary`: visitedCount, emittedCount, maximumDepthVisited, + scopeRootCount, deduplicatedCount, prunedSubtrees, depthLimitedSubtrees, completion. +- `JSONTraversalCompletion`: complete, restricted, visitorStopped. +- `JSONTraversalFailure`: collectionLimitExceeded(limit:), ambiguousRoots(first:second:). + +The [DocC guide](../../../Sources/JSONNavigation/Documentation.docc/Traversal.md) +contains three externally compiled/executed before/after examples. The actual +[consumer](../../../Tools/NavigationQualification/Consumer.swift) uses public JSON +and Navigation declarations, not package meters. Its external-module compiler +probes check core import isolation, hidden meters, non-Sendable cursor rejection +and absence of a fabricated missing kind in both build configurations. + +## Exactness and coverage + +DFS preorder is the only order offered: object insertion and increasing array +positions. Depth counts edges from each selected root. Full source admission is +mandatory even for a scalar selection, depth zero, filtered-empty selection or +early stop. Root omission/kind filtering changes emission only. Pruning/depth +omission marks coverage restricted, and explicit stop marks visitorStopped. Errors +and cancellation throw. Reports are conservative per-occurrence coverage, not a +union-coverage proof or a global absence query. + +All virtual roots resolve before any callback. Reject mode then checks every pair +for equal or ancestor/descendant typed prefixes. Ordered mode preserves root order, +then each subtree's DFS, including repeats. Unique mode remembers **first examined** +typed paths, even excluded/filtered roots, preserving their first depth/origin. It +still explores revisited descendants under the current occurrence's depth policy. +Two equal-valued siblings or scalar-distinct keys never merge. A virtual selection +does not invent a new source root, array index, durable ID or revision fingerprint. + +Mapped spans are opt-in core lookups, never synthesized. An unchanged captured old +document continues to describe its old root after another traversal captures a +replacement. No record resolves against a caller-supplied newer root automatically. + +## Finite policy and implementation costs + +All 49 core metrics/profiles/weights remain unchanged. Existing core read-scope +admission covers root, source bytes/label and source maps. Path records reuse its +pointer, diagnostic and retention policies. The N1 logical overlay reserves 128 +bytes for summary/staging, 16 for root-list staging, 64 per planned root or frontier +frame, 64 per ephemeral hit or collected record, and 16 per unique-path reference. +Retained collected paths additionally charge the published 32-byte indexSlot per +component and its key UTF-8 bytes. These are logical weights, not allocation/RSS +measurements; caller-retained callback copies remain caller storage. + +Collection has an independent inclusive default of 100,000 records; zero prohibits +emission but allows a genuinely empty result. Negative bounds/depth reject as +configuration errors. Its next-record refusal throws a typed traversal capacity +error and publishes no collection. Source admission uses existing node capacities; +visits do not misleadingly re-charge source nodes, and work stays cumulative across +roots, pair scans, dedup, emitted paths, source-span lookups, output and cleanup. + +Initial reject/unique modes use exact metered linear comparisons. Pair validation +and unique-node scans can be quadratic. Core path snapshots along a deep frontier +can also use quadratic path metadata. The prototype intentionally makes no +linear-total-time, indexed-lookup, zero-copy or speed claim. Every owned comparison +loop is cancellable/budgeted, so exhaustion is an operational error, never a partial +success. A future compact representation/index needs its own measured review. + +The invocation and its synchronous scope never escape into a callback. JSONTraversal, +metadata/options and collected values are checked Sendable; each simultaneous call +creates its own scope. A package-only nil-by-default checkpoint observer supports +owned-loop cancellation qualification; no public consumer can install it. + +## Qualification and promotion boundary + +The reviewed new inventory is 36 nonparameterized Navigation tests, including the +three pinned synthetic packet documents. Existing function identities and all 325 +native/Linux parameter invocations remain; WASI retains its separate 288. Full roots +become 582 native, 566 Linux and 478 WASI, and shared-state counts 42/41. All 126 +maintenance identities, corpus policies and 49 cap probes remain required. + +`Scripts/ci.py --lane navigation` checks exact focused discovery, Debug/Release, +explicit strict-memory compilation and external compiler/consumer/DocC probes. +Native/Linux run the new consumer and full existing suites; safety includes Navigation +ASan plus its shared-root TSan function. WASI uses the updated exact root inventory. +Current commands/receipts, failures and platform gaps accompany the return; this +decision document does not assert a pending platform run passed. Full release-gate +qualification and explicit product review precede promotion to an installable product. + +BFS, scalar-sorted order, prefix snapshots, JSONInspection, classifiers, snapshots, +semantic/durable digests, projections, editing, retrieval, actors, rendering and CLI +verbs are explicitly deferred. No EngineBench implementation or performance claim. diff --git a/Documentation/Architecture/StructuralIntelligence/Progress.json b/Documentation/Architecture/StructuralIntelligence/Progress.json new file mode 100644 index 0000000..384e879 --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/Progress.json @@ -0,0 +1,12 @@ +{ + "schema_version": 1, + "research_base": "507dbf9ad4b545a5d60d7d6acf6b814795ec3099", + "implementation_base": "037e735dc6224789e4d04d2c18d15b8d27440198", + "packet_sha256": "d758f75d84271a97da515bf7b7ef70de7a61e91fdf949a511fe0fca2708156f8", + "N0": "source authorities reconciled; core-only seven-witness consumer and package-scoped one-ledger composition compiled/executed", + "N1": "experimental physical/virtual DFS, bounded collection and 36 focused behavior functions implemented; complete platform qualification in progress", + "public_product": false, + "public_products_preserved": 7, + "core_semantic_changes": false, + "deferred": ["BFS", "N2", "N3", "P1", "P2", "R1", "R2", "actors", "persistence", "query grammar", "rendering", "CLI verbs", "EngineBench"] +} diff --git a/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md b/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md new file mode 100644 index 0000000..4ae2b3c --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md @@ -0,0 +1,63 @@ +# N0 source authority + +The research baseline is `507dbf9ad4b545a5d60d7d6acf6b814795ec3099`. +The reconciled implementation base is main +`037e735dc6224789e4d04d2c18d15b8d27440198`, tree +`5c53288c50ae8a1999d889f0a8c4f0cb14dca913`. Origin was fetched before edits. +PR #33 forward-ports maintenance repairs after the research snapshot; no reset, +old scaffold or historical navigation implementation is used. + +At reconnaissance all nine main checks passed: Native CI 38735450692 and Linux +CI 38735450703. Those are baseline results, not qualification of this increment. +The latest published release observed was 1.5.0; main's `1.6.0-dev` CLI metadata +does not establish a publication. Draft #31 owns opt-in value/schema DSLs, and a +separate schema-example worktree has active edits. Neither is imported or modified. +The primary checkout stays on main; implementation uses +`codex/structural-navigation-n0-n1` in an isolated existing feature worktree. + +## Semantic delegates and owners + +Paths below identify code ownership, not a second resolver API. All new owned +phases use the **same** read-scope state and its noncopyable core-domain meter. + +| New operation or fact | Existing authority / owner | Budget boundary | +| --- | --- | --- | +| Captured canonical source | [JSONDocument](../../../Sources/JSON/Document/JSONDocument.swift) and JSON | Constructor captures trusted immutable values; each invocation admits its full root/source/map | +| Admission and source lifetime | [JSONReadScope.init/check/complete/invalidate](../../../Sources/JSON/Reading/JSONReadScope.swift) | Exactly one root/source admission and terminal ledger; complete releases scope ownership | +| Exact keys / insertion order | [JSONObject](../../../Sources/JSON/Value/JSONObject.swift), JSONKey | Visit ordinal members; no String sort, dictionary conversion or duplicate index | +| Typed physical selection | JSONReadScope.select inside run | Strict key/index parent checks and existing JSONAccessFailure; no Pointer-token reinterpretation | +| Typed path staging | JSONReadScope.append and JSONReadPath | Existing pointerTokens/pointerBytes/diagnosticBytes checks, path copy/UTF-8 work and live refunds | +| Meter/operational failure | [JSONWorkMeter](../../../Sources/JSON/Limits/JSONWorkMeter.swift), JSONCodingState | Shared cumulative work, capacity/retention, cancellation and terminal health; never reset | +| DFS child enumeration | JSON array indices / JSONObject ordinal collection access | New iterative frontier; each node/edge/frame/callback handoff checked and charged | +| Overlap / unique-path comparison | [JSONComparison.textEquals](../../../Sources/JSON/Value/JSONComparison.swift) + typed integer components | New metered component/prefix loops; no Set, unmetered path hash, or value-based dedup | +| Decimal and text values given to visitor | JSON/JSONNumber/JSONKey | Original source value; no conversion or re-admission per hit | +| Pointer serialization / differential oracle | [JSONPointer](../../../Sources/JSON/Pointer/JSONPointer.swift) | Explicit separate caller operation; walker never calls a public helper to start a fresh owned quota | +| Native projection / presence oracle | [JSONReadCursor](../../../Sources/JSON/Reading/JSONReadCursor.swift), JSONPresence | Compiler/behavior witnesses only; no cursor is a traversal result or task argument | +| Optional value spans | [JSONSourceMap.span](../../../Sources/JSON/Document/JSONSourceMap.swift) | Same meter, only coherent captured maps; replacingRoot supplies no map | +| Successful resource reporting | JSONOperationReport / JSONOperationStatistics | Existing accountingVersion/domain/counts/peak convention; no successful report after error | +| Mutations | JSONPatch / JSONMergePatch | Unchanged and outside N1; traversal has no edits or writable value view | + +`JSONReadScope` and `JSONWorkMeter` are already `package`, so no core SPI, +public meter protocol or package helper change is needed. Actual target compilation +and the inclusive work-budget tests prove this access/composition boundary. The +public N0 consumer compiled and ran seven core-only witnesses before N1: root versus +empty key, typed key/index, exact Unicode keys, presence, native projection and +invalid array-index rejection. Compiler receipts retain the actual source/toolchain. + +## Collision and dialect reconciliation + +| Historical/proposed surface | Disposition | +| --- | --- | +| JSONCursor / JSONNavigationCursor | Rejected: guarded core read cursor already owns synchronous one-node access | +| JSONNodePath / dotted flattening | Rejected: canonical components are JSONLocationComponent; root [] and empty key differ | +| New JSON. facade / JSONPath grammar | Rejected: neither necessary nor an RFC 9535 implementation | +| JSONFilterRuntime paths/getpath/walk | Existing private query dialect can accept negative indices and language-level missing behavior; not a strict typed scope delegate | +| JSONHTML exact paths and previews | Existing rendering authority; no renderer import/edit or duplicate preview logic | +| JSONTraversal and metadata names | New target-qualified names; no existing core/sibling definition found at the recorded base | +| Sketch BFS / inspection / snapshots / projections | Deferred; no reserved BFS label, parallel model, digest or classifier added | + +The packet's declarations were illustrative. N1 returns existing +JSONOperationReport envelopes, passes the source value separately to a synchronous +visitor, and keeps records metadata-only. Its module is experimental; **no public +SwiftPM product is added**. API shape and all remaining release gates are recorded +in [API-DECISIONS](API-DECISIONS.md). diff --git a/Documentation/CI/README.md b/Documentation/CI/README.md index 04cb04d..2e50aeb 100644 --- a/Documentation/CI/README.md +++ b/Documentation/CI/README.md @@ -1,5 +1,17 @@ # Native CI +## Experimental N1 branch + +The Structural Intelligence branch adds exactly 36 Navigation functions and one +shared-root safety function, keeping every existing identity/case. Root expectations +are 582 native/566 Linux with 325 parameter invocations; TSan is 42/41. WASI's +separate core scope is 478 functions/288 parameters. No new public product is exposed. +`--lane navigation` checks focused discovery, Debug/Release, explicit strict-memory +compilation and the external compiler/consumer/experimental DocC audit. Native/Linux +include that consumer alongside their existing gates; all 49 core cap outcomes and +126 restored maintenance identities remain required. The following earlier counts +retain their original source context; actual source-bound receipts establish passes. + This main-based forward-port retains seven products and restores the entire 1.2.1 regression inventory. The experimental composition increment adds 12 functions; current native root discovery is 558 functions and 325 parameter invocations; Linux is 542/325. Full ASan uses the same roots; diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json index 9ff271b..805b48a 100644 --- a/Documentation/Qualification/TestDiscovery.json +++ b/Documentation/Qualification/TestDiscovery.json @@ -9,7 +9,8 @@ "JSONIntegrationTests": 32, "JSONHTMLTests": 97, "JSONBridgesTests": 10, - "JSONCLITests": 88 + "JSONCLITests": 88, + "JSONNavigationTests": 36 }, "parameterized_invocations": 325, "composition_external_functions": 5, @@ -29,7 +30,8 @@ "JSONIntegrationTests": 11, "JSONHTMLTests": 5, "JSONBridgesTests": 1, - "JSONCLITests": 1 + "JSONCLITests": 1, + "JSONNavigationTests": 1 }, "coverage_unmapped": { "Sources/JSON/Pointer/JSONLocationComponent.swift": "declaration-only enum", @@ -53,7 +55,8 @@ "JSONBridges", "JSONFilterRuntime", "JSONCLIEngine", - "JSONCLI" + "JSONCLI", + "JSONNavigation" ], "portable_root_functions": { "JSONTests": 190, @@ -64,7 +67,8 @@ "JSONIntegrationTests": 32, "JSONHTMLTests": 81, "JSONBridgesTests": 10, - "JSONCLITests": 88 + "JSONCLITests": 88, + "JSONNavigationTests": 36 }, "wasi_root_functions": { "JSONTests": 190, @@ -74,7 +78,8 @@ "JSONPatchTests": 16, "JSONIntegrationTests": 32, "JSONHTMLTests": 81, - "JSONBridgesTests": 10 + "JSONBridgesTests": 10, + "JSONNavigationTests": 36 }, "wasi_parameterized_invocations": 288, "portable_shared_state_functions": { @@ -85,7 +90,8 @@ "JSONIntegrationTests": 11, "JSONHTMLTests": 4, "JSONBridgesTests": 1, - "JSONCLITests": 1 + "JSONCLITests": 1, + "JSONNavigationTests": 1 }, "export_functions": { "JSONSchemaTests": 1, diff --git a/Documentation/README.md b/Documentation/README.md index c1b2b26..2c7ee0d 100644 --- a/Documentation/README.md +++ b/Documentation/README.md @@ -41,6 +41,8 @@ Online references track main; tagged releases include matching compiled catalogu ## Consumers, qualification and maintenance +- [Experimental N0/N1 source authority](Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md) and [gated traversal decisions](Architecture/StructuralIntelligence/API-DECISIONS.md); JSONNavigation is not a released product. + - [Six-library compiled example](../Examples/Consumer/README.md) - [Downstream JSON/JSONSchema interface](ConsumerInterface.md) and [runnable public consumer](../Tools/MCPReadiness/README.md) - [Independent-suite policies and qualification](Qualification/README.md) diff --git a/Package.swift b/Package.swift index c494a96..4170c9c 100644 --- a/Package.swift +++ b/Package.swift @@ -16,6 +16,9 @@ let package = Package( ], targets: [ .target(name: "JSON"), + // Experimental N1 target. Product promotion requires separate release-gate review. + .target(name: "JSONNavigation", dependencies: ["JSON"]), + .testTarget(name: "JSONNavigationTests", dependencies: ["JSONNavigation", "JSON"], exclude: ["Fixtures"]), .target(name: "JSONBridges", dependencies: ["JSON"]), .testTarget(name: "JSONIntegrationTests", dependencies: ["JSON", "JSONSchema", "JSONStreaming", "JSONPatch"]), .testTarget(name: "JSONBridgesTests", dependencies: ["JSONBridges", "JSON"]), diff --git a/README.md b/README.md index bbd9c15..4005f31 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,11 @@ Maintained by [Thorsten Fuchs](https://github.com/ThorFuchs). Swift-JSON follows semantic versioning. Use a released version and retain `Package.resolved` for reproducible builds; `main` tracks ongoing development. +This feature branch also contains an [experimental structural traversal target](Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md). +It preserves the seven released products. JSONNavigation product promotion remains +gated; the [compiled traversal guide](Sources/JSONNavigation/Documentation.docc/Traversal.md) +describes its many-node scope, exact locations, finite accounting and coverage. + Published **1.5.0** provides six libraries and the CLI. This development branch retains main’s unreleased structured HTML previews and forward-ports the 1.5.1 maintenance repairs: best-effort schema diagnostics, the complete 1.2.1 regression diff --git a/Scripts/README.md b/Scripts/README.md index f491eb5..26c3e7a 100644 --- a/Scripts/README.md +++ b/Scripts/README.md @@ -9,6 +9,7 @@ separate from the dependency-free library. Native lanes fail on unsupported host | `python3 Scripts/preflight.py` | Actual native OS/compiler/SDK receipt | | `python3 Scripts/ci.py --lane native` | Manifests, builds, Swift Testing, consumers and suites | | `python3 Scripts/ci.py --lane safety` | Address/Thread Sanitizer and strict memory safety | +| `python3 Scripts/ci.py --lane navigation` | Experimental N1 exact discovery, Debug/Release, strict-memory and external compiler/consumer/DocC proof; no product promotion | | `python3 Scripts/ci.py --lane linux` | Actual Linux preflight, all seven products, exact root/cap/corpus/consumer counts | | `python3 Scripts/ci.py --lane linux-safety` | Linux Address/Thread Sanitizer and strict memory safety | | `python3 Scripts/ci.py --lane wasm` | Matched Swift 6.4 WASI debug/release, exact root/corpus/cap/consumer counts, two-host public parity and structured resource failures | diff --git a/Scripts/ci.py b/Scripts/ci.py index b6a6478..309c2fa 100644 --- a/Scripts/ci.py +++ b/Scripts/ci.py @@ -11,6 +11,7 @@ SHARED_TESTS = 'independentCodecFacadesAndExactValuesCanBeSharedAcrossTasks|concurrentSchemaEvaluationsHaveExclusiveDynamicRegexAndAnnotationState|sharedRegexProgramHasExclusiveConcurrentSearchState|immutableIndexedObjectCanBeSharedAcrossTasks|qaSharedPayloadAndRootHaveIndependentConcurrentReadScopes|sha256InputCapacityAndConcurrentInvocationsRemainIndependent|sharedFormattedHTMLRendererHasIndependentConcurrentInvocations|sharedAnnotatedDocumentRendererHasIndependentConcurrentInvocations|sharedExportPreparationsHaveIndependentConcurrentState|applePDFSimultaneousJobsHaveIndependentStateAndCancellation|sharedFilterPlanHasIndependentConcurrentInvocations|structuredSharedRendererPlansHaveInvocationOwnedState|sharedProviderBridgeHasIndependentConcurrentInvocations|e2SharedBridgeInputsHaveInvocationOwnedState|e2SharedCompiledPlanFacadeHasIndependentInvocationState' SHARED_TESTS += '|compositionIndependentScopesCanBeSharedAcrossTasks' SHARED_TESTS += '|qaStreaming|qaE2E(SharedPlan|RootImport|NDJSONImport|AwaitedSink|ExactWorkflow)' +SHARED_TESTS += '|navigationSharedImmutableRootIndependentInvocations' R0_TESTS = 'readerProbe|indexProbes|duplicateRecognitionPrecedesValueAndClosingSyntax' def snapshot(): @@ -32,6 +33,13 @@ def commands(lane, output): ['python3', '-B', 'Scripts/qualify_composition.py', '--output', str(output / 'compiler-negative')], ] return [c + ['--disable-xctest'] if sys.platform == 'linux' and c[:2] == ['swift', 'test'] else c for c in result] + if lane == 'navigation': + return [['python3', '-B', 'Scripts/public_check.py'], + ['swift', 'test', '--filter', 'navigation'], + ['swift', 'test', '-c', 'release', '--filter', 'navigation'], + ['swift', 'build', '--target', 'JSONNavigation', '--scratch-path', '.build/navigation-strict', + '-Xswiftc', '-strict-memory-safety', '-Xswiftc', '-warnings-as-errors'], + ['python3', '-B', 'Scripts/qualify_navigation.py', '--output', str(output / 'consumer')]] if lane == 'archive': return [['python3', '-B', 'Scripts/public_check.py'], ['python3', '-B', 'Scripts/qualify_native_reproduction.py', '--output', str(output / 'reproduction')]] @@ -112,6 +120,7 @@ def commands(lane, output): ['swift', 'run', '-c', 'release', '--package-path', 'Examples/Consumer', 'Consumer'], ['python3', '-B', 'Scripts/qualify_distribution.py', '--output', str(output / 'distribution')], ['python3', '-B', 'Scripts/qualify_ergonomics.py'], + ['python3', '-B', 'Scripts/qualify_navigation.py', '--output', str(output / 'navigation')], ['swift', 'test', '--package-path', 'Tools/MobileQualification'], ['swift', 'run', '-c', 'release', '--package-path', 'Tools/MobileQualification', 'MobileQualification'], ['swift', 'test', '--package-path', 'Tools/MCPReadiness'], @@ -160,6 +169,10 @@ def check_result(command, code, log, events=None): if events is not None: counts = events['bundle_counts'] if command[:2] == ['swift', 'test']: expected = [DISCOVERY['composition_external_functions']] if 'Tools/CompositionQualification' in command else [12] if 'composition|composed|caughtOperationalFailureCannotComplete' in command else [88] if 'cli|sharedFilterPlan' in command else sorted(DISCOVERY['export_functions'].values()) if 'export|Export|apple' in command else [14] if 'Tools/WasmAudit' in command else sorted(DISCOVERY['qa_functions'].values()) if 'qa' in command else [7] if R0_TESTS in command else [3] if 'Tools/MCPReadiness' in command or 'Tools/MobileQualification' in command else [1] if 'Tools/Qualification' in command else sorted(SHARED_COUNTS.values()) if 'thread' in command else sorted(ROOT_COUNTS.values()) + if 'navigation' in command: + expected = [DISCOVERY['root_functions']['JSONNavigationTests']] + if events is not None: + passed = passed and events['module_function_counts'] == {'JSONNavigationTests': expected[0]} and events['parameter_case_invocations'] == 0 passed = passed and sorted(counts) == expected if 'Tools/CompositionQualification' in command and events is not None: passed = passed and events['module_function_counts'] == {'CompositionScaleTests': DISCOVERY['composition_external_functions']} and events['parameter_case_invocations'] == 0 @@ -200,7 +213,7 @@ def check_result(command, code, log, events=None): passed = sorted(p['name'] for p in d['products']) == ['JSON', 'JSONBridges', 'JSONHTML', 'JSONPatch', 'JSONSchema', 'JSONStreaming', 'swift-json'] and not d['dependencies'] and d['swiftLanguageVersions'] == ['6'] and d['toolsVersion']['_version'] == '6.4.0' and d['platforms'] == [{'options': [], 'platformName': 'macos', 'version': '27.0'}, {'options': [], 'platformName': 'ios', 'version': '27.0'}] passed = passed and {p['name']: p['targets'] for p in d['products']} == dict({name: [name] for name in ['JSON', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONBridges']}, **{'swift-json': ['JSONCLI']}) targets = {t['name']: t for t in d['targets']} - passed = passed and set(targets) == {'JSON', 'JSONIntegrationTests', 'JSONBridges', 'JSONBridgesTests', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONRegexRuntime', 'JSONTests', 'JSONSchemaTests', 'JSONStreamingTests', 'JSONPatchTests', 'JSONHTMLTests', 'JSONRegexRuntimeTests', 'JSONFilterRuntime', 'JSONCLIEngine', 'JSONCLI', 'JSONCLITests'} and targets['JSONHTML']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONBridges']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONFilterRuntime']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONCLI']['type'] == 'executable' + passed = passed and set(targets) == {'JSON', 'JSONNavigation', 'JSONNavigationTests', 'JSONIntegrationTests', 'JSONBridges', 'JSONBridgesTests', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONRegexRuntime', 'JSONTests', 'JSONSchemaTests', 'JSONStreamingTests', 'JSONPatchTests', 'JSONHTMLTests', 'JSONRegexRuntimeTests', 'JSONFilterRuntime', 'JSONCLIEngine', 'JSONCLI', 'JSONCLITests'} and targets['JSONHTML']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONBridges']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONNavigation']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONNavigationTests']['dependencies'] == [{'byName': ['JSONNavigation', None]}, {'byName': ['JSON', None]}] and targets['JSONFilterRuntime']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONCLI']['type'] == 'executable' if command == ['swift', 'package', 'show-dependencies', '--format', 'json'] and passed: passed = json.loads(log)['dependencies'] == [] if command[:2] == ['swift', 'run'] and 'CompositionConsumer' in command: @@ -213,6 +226,8 @@ def check_result(command, code, log, events=None): passed = passed and 'TESTING_EVENT_RECEIPTS passed cases=19' in log.splitlines() if command[:3] == ['python3', '-B', 'Scripts/qualify_ergonomics.py']: passed = passed and 'ERGONOMICS_COMPILER passed commands=8 guide_fences=3' in log.splitlines() + if command[:3] == ['python3', '-B', 'Scripts/qualify_navigation.py']: + passed = passed and 'NAVIGATION_COMPILER passed configurations=2 consumer_cases=18 negative_probes=8' in log.splitlines() if command[:3] == ['python3', '-B', 'Scripts/verify_html_assets.py']: passed = passed and 'JSONHTML_ASSETS passed count=2' in log.splitlines() if 'BrowserFixtures' in command: @@ -225,7 +240,7 @@ def check_result(command, code, log, events=None): def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('--lane', choices=['composition', 'native', 'safety', 'docs', 'r0', 'qa', 'linux', 'linux-safety', 'wasm', 'browser', 'export', 'cli', 'coverage', 'linux-coverage', 'linux-qa', 'wasm-qa', 'archive'], required=True) + parser.add_argument('--lane', choices=['composition', 'native', 'safety', 'docs', 'r0', 'qa', 'linux', 'linux-safety', 'wasm', 'browser', 'export', 'cli', 'coverage', 'linux-coverage', 'linux-qa', 'wasm-qa', 'archive', 'navigation'], required=True) parser.add_argument('--output') args = parser.parse_args() output = ROOT / (args.output or '.build/ci/' + args.lane) diff --git a/Scripts/qualify_navigation.py b/Scripts/qualify_navigation.py new file mode 100644 index 0000000..57bced7 --- /dev/null +++ b/Scripts/qualify_navigation.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Compile/run the unpublished N1 module as an external consumer, including negative compiler probes.""" +import argparse +import hashlib +import json +import pathlib +import re +import shutil +import subprocess +import sys + +ROOT = pathlib.Path(__file__).resolve().parents[1] +CHECKS = {'root-empty-key', 'typed-key-index', 'presence', 'scalar-keys', 'native-exact', + 'pointer-parity', 'metadata-collection', 'ordered-occurrences', 'unique-paths', + 'default-overlap-rejection', 'typed-missing', 'complete-empty-filter', + 'collection-limit', 'cumulative-report', 'coherent-spans', + 'replacement-invalidates-spans', 'intentional-stop', 'cumulative-quota'} + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--output', required=True) + args = parser.parse_args() + output = ROOT / args.output + output.mkdir(parents=True, exist_ok=True) + receipt = {'schema_version': 1, 'status': 'running', 'commands': [], + 'scope': 'External Swift module API proof; JSONNavigation is not a released SwiftPM product', + 'consumer_sha256': hashlib.sha256((ROOT / 'Tools/NavigationQualification/Consumer.swift').read_bytes()).hexdigest()} + + def run(name, command, expected=None): + result = subprocess.run(command, cwd=ROOT, capture_output=True, text=True, timeout=900) + log = output / (name + '.log') + log.write_text((result.stdout + result.stderr).replace(str(ROOT), '.')) + passed = result.returncode == 0 if expected is None else result.returncode != 0 and expected in result.stderr + receipt['commands'].append({'name': name, 'command': [x.replace(str(ROOT), '.') for x in command], + 'exit_code': result.returncode, 'status': 'passed' if passed else 'failed', + 'expected_diagnostic': expected, 'log': log.relative_to(ROOT).as_posix(), + 'log_sha256': hashlib.sha256(log.read_bytes()).hexdigest()}) + (output / 'receipt.json').write_text(json.dumps(receipt, indent=2) + '\n') + if not passed: + raise ValueError('Compiler/consumer outcome failed: ' + name) + return result.stdout + + try: + for config in ['debug', 'release']: + run(config + '-build', ['swift', 'build', '-c', config, '--target', 'JSONNavigation']) + # Build settings are interrogated, never inferred from a user's checkout name. + binpath = pathlib.Path(run(config + '-binpath', ['swift', 'build', '-c', config, '--show-bin-path']).strip()) + modules = binpath / 'Modules' if (binpath / 'Modules').is_dir() else binpath + objects = [] + for module in ['JSON', 'JSONNavigation']: + aggregate = binpath / (module + '.o') + selected = [aggregate] if aggregate.is_file() else sorted((binpath / (module + '.build')).glob('*.o')) + if not selected: + raise ValueError('Missing built module objects: ' + module) + objects.extend(str(p) for p in selected) + common = ['swiftc', '-swift-version', '6', '-warnings-as-errors', '-I', str(modules), '-package-name', 'navigation_external_consumer'] + executable = output / ('consumer-' + config) + run(config + '-consumer-compile', common + ['-parse-as-library', str(ROOT / 'Tools/NavigationQualification/Consumer.swift'), *objects, '-o', str(executable)]) + text = run(config + '-consumer-run', [str(executable)]) + observed = re.findall(r'^NAVIGATION_CHECK (\S+) passed$', text, re.M) + if len(observed) != 18 or set(observed) != CHECKS or 'NAVIGATION_CONSUMER passed cases=18' not in text: + raise ValueError('Missing/duplicate public consumer outcomes') + resource = re.search(r'^NAVIGATION_RESOURCE work=(\d+) retained=(\d+) peak=(\d+) nodes=(\d+)$', text, re.M) + if resource is None or int(resource[4]) != 10 or not all(int(n) > 0 for n in resource.groups()): + raise ValueError('Missing cumulative resource observations') + receipt.setdefault('consumer_results', {})[config] = {'cases': observed, 'work_units': int(resource[1]), + 'logical_retained_bytes': int(resource[2]), 'peak_logical_retained_bytes': int(resource[3]), 'admitted_nodes': int(resource[4])} + guide = ROOT / 'Sources/JSONNavigation/Documentation.docc/Traversal.md' + fences = re.findall(r'```swift\n(.*?)\n```', guide.read_text(), re.S) + if len(fences) != 3: + raise ValueError('Changed three-fence compiled guide inventory') + for index, fence in enumerate(fences): + file = output / ('guide-' + str(index) + '.swift') + file.write_text('import JSON\nimport JSONNavigation\n@main struct Guide { static func main() throws {\n' + fence + '\n} }\n') + binary = output / ('guide-' + config + '-' + str(index)) + run(config + '-guide-' + str(index) + '-compile', common + ['-parse-as-library', str(file), *objects, '-o', str(binary)]) + run(config + '-guide-' + str(index) + '-run', [str(binary)]) + probes = [ + ('core-import-isolation', 'import JSON\nlet x = JSONTraversal.self\n', 'cannot find'), + ('private-meter-isolation', 'import JSON\nlet x = JSONWorkMeter.self\n', 'cannot find'), + ('cursor-cross-task-rejection', 'import JSON\nfunc cross(_ cursor: JSONReadCursor) { let f: @Sendable () throws -> JSON = { try cursor.requiredValue() }; _ = f }\n', 'non-Sendable'), + ('no-missing-kind', 'import JSONNavigation\nlet x = JSONTraversalKind.missing\n', 'has no member'), + ] + for name, source, expected in probes: + file = output / (name + '.swift') + file.write_text(source) + run(config + '-' + name, common + ['-typecheck', str(file)], expected) + if sys.platform == 'darwin': + graphs = output / 'symbols'; graphs.mkdir(exist_ok=True) + # Standalone extraction from an optimized serialized module omits source + # locations. SwiftPM emits source-aware graphs during real compilation. + run('symbol-graph', ['swift', 'package', 'dump-symbol-graph', '--minimum-access-level', 'public']) + for path in (ROOT / '.build/out/symbolgraph').glob('JSONNavigation*.symbols.json'): + shutil.copyfile(path, graphs / path.name) + import documentation_catalog + # This local audit does not promote an experimental module to the public site. + documentation_catalog.PRODUCTS = ('JSONNavigation',) + coverage = documentation_catalog.verify_comments(documentation_catalog.public_symbols(graphs)) + receipt['experimental_api_coverage'] = coverage + if coverage['errors']: + raise ValueError('Undocumented experimental API: ' + str(coverage['errors'])) + run('docc', ['xcrun', 'docc', 'convert', str(ROOT / 'Sources/JSONNavigation/Documentation.docc'), + '--additional-symbol-graph-dir', str(graphs), '--output-dir', str(output / 'JSONNavigation.doccarchive'), + '--fallback-bundle-identifier', 'org.swift-json.experimental.JSONNavigation', '--warnings-as-errors']) + receipt['status'] = 'passed' + except (OSError, ValueError, subprocess.TimeoutExpired) as error: + receipt['status'] = 'failed'; receipt['error'] = str(error) + (output / 'receipt.json').write_text(json.dumps(receipt, indent=2) + '\n') + print('NAVIGATION_COMPILER ' + receipt['status'] + ' configurations=2 consumer_cases=18 negative_probes=8') + return 0 if receipt['status'] == 'passed' else 1 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift index a8cb76f..9cb352a 100644 --- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift +++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift @@ -2,6 +2,6 @@ // Generated by Scripts/generate_cli_identity.py; excludes this file. // SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit. package enum CLIBuildIdentity { - package static let revision = "sha256:6da72a4288423d431dd46acd076fc9ee2f4a5cc68a5ea8bf55e31bd9b0b7b1ea" + package static let revision = "sha256:21ad62ecd2a10b747b0b45105f85251213e8d49dae0a9c05fabbbfb4c46868ff" package static let kind = "source-sha256" } diff --git a/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md b/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md new file mode 100644 index 0000000..9e65c2d --- /dev/null +++ b/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md @@ -0,0 +1,32 @@ +# ``JSONNavigation`` + +Exact, synchronous, bounded exploration of many nodes in one canonical JSON source. + +This is an experimental N1 target, **not a released SwiftPM product**. The package's +seven public products remain unchanged. Product promotion requires the recorded +correctness, resource, compiler, platform and API review gate. + +The core JSON module continues to own one-node lookup, exact numbers and keys, +presence and RFC 6901 Pointer. Use ``JSONTraversal`` for deterministic DFS preorder, +physical subtrees and explicitly ordered virtual selections. One call admits the +captured source once and shares its core read-scope meter across all phases. + +## Topics + +### Exact structural exploration + +- +- ``JSONTraversal`` +- ``JSONTraversalScope`` +- ``JSONTraversalOptions`` +- ``JSONScopeOverlapPolicy`` + +### Observation and coverage + +- ``JSONTraversalHit`` +- ``JSONTraversalKind`` +- ``JSONTraversalAction`` +- ``JSONTraversalSummary`` +- ``JSONTraversalCompletion`` +- ``JSONTraversalCollection`` +- ``JSONTraversalFailure`` diff --git a/Sources/JSONNavigation/Documentation.docc/Traversal.md b/Sources/JSONNavigation/Documentation.docc/Traversal.md new file mode 100644 index 0000000..b04f9bc --- /dev/null +++ b/Sources/JSONNavigation/Documentation.docc/Traversal.md @@ -0,0 +1,76 @@ +# Exact traversal and explicit coverage + +N1 adds many-node observation to the existing exact model. These three examples +are compiled and executed as external Swift 6 modules in Debug and Release by +`Scripts/qualify_navigation.py`. They assume explicit `import JSON` and +`import JSONNavigation`; installing a public Navigation product remains gated. + +Core access remains the appropriate interface for one node: + +```swift +let root = try JSON.parse(#"{"items":[1,2],"note":null}"#) +let selected = try JSONPointer(parsing: "/items/0").resolve(in: root) +let absent = try root.presence(of: "missing") +let null = try root.presence(of: "note") +precondition(selected == 1 && absent == .absent && null == .null) +``` + +The structural visitor explores object insertion order and ascending array indices. +Its paths remain absolute typed components; numeric keys remain keys. Root is `[]` +and empty-key is `[.key("")]`; Pointer text should be produced by the core Pointer. + +```swift +let root = try JSON.parse(#"{"items":[1,2]}"#) +var locations: [[JSONLocationComponent]] = [] +let report = try JSONTraversal(document: .init(root: root)).visit { hit, _ in + locations.append(hit.absoluteLocation) + return .continue +} +precondition(locations == [[], [.key("items")], [.key("items"), .index(0)], [.key("items"), .index(1)]]) +precondition(report.result.completion == .complete && report.statistics[.nodes] == 4) +``` + +Virtual scopes select existing source nodes without materializing a new JSON array. +The default rejects duplicate and ancestor/descendant roots before any callback. + +```swift +let root = try JSON.parse(#"{"items":[1,2],"tail":null}"#) +let selection = JSONTraversalScope.virtual(roots: [[.key("items")], [.key("tail")]]) +let report = try JSONTraversal(document: .init(root: root)).collect(scope: selection, maximumRecords: 4) +precondition(report.result.nodes.count == 4 && report.result.summary.scopeRootCount == 2) +precondition(report.result.summary.completion == .complete) +``` + +`.orderedOccurrences` preserves repeated selections and their root index, occurrence +ordinal and relative depth. `.uniquePhysicalNodes` deduplicates exact typed paths, +including filtered/excluded roots: the first examined occurrence wins. Equal values +at different paths remain different nodes. Revisited paths still explore descendants +under their current relative depth; deduplication is never automatic subtree pruning. + +Depth zero permits only a selected root. `includeScopeRoot: false` suppresses its +emission; `depthLimit: 1` then gives children-only traversal. Kind filtering suppresses +emission, never descendant exploration. A filtered empty result can be complete; +a missing scope throws. Pruning/depth omission is `.restricted`, explicit `.stop` is +`.visitorStopped`; neither establishes absence outside observed coverage. Coverage is +conservative per occurrence, not a proof that overlapping selections repaired an +earlier pruned region. Only DFS is implemented; no BFS label or prefix-snapshot API. + +Collection defaults to 100,000 records and throws on the next record; its bound is +inclusive and zero is meaningful. Root/source admission, scope resolution, exact +overlap comparisons, path creation, frontier, source-span lookup and records share +one finite core ledger. Source spans are opt-in and come only from the captured +coherent document. `replacingRoot` supplies none. No source text or subtree appears +in a hit, and no read cursor crosses the synchronous visitor boundary. + +Caller visitor computation and caller-retained copies are outside owned instruction +and retention quotas. Reports account for logical ownership, not process RSS or +allocator capacity. Unexpected quotas and cancellation throw and return no successful +report, even if cancellation occurs during a callback returning `.stop`. Earlier +callback effects cannot be recalled. Core access/resource errors retain their types; +collection capacity and ambiguous roots use ``JSONTraversalFailure``. + +The initial overlap/dedup implementation uses metered linear comparisons rather than +an unqualified hash index. Complete path copies are bounded by the existing read +scope; deep frontiers can use quadratic path metadata. No EngineBench speed, linear +total-time, zero-copy or fixed-memory claim is made. Classification, snapshots, +projections, persistence, async actors, query syntax and rendering are later gates. diff --git a/Sources/JSONNavigation/JSONTraversal.swift b/Sources/JSONNavigation/JSONTraversal.swift new file mode 100644 index 0000000..9ef4a8e --- /dev/null +++ b/Sources/JSONNavigation/JSONTraversal.swift @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON + +/// Synchronous exact structural exploration of one immutable canonical document. +/// +/// Each call admits the complete captured root/source once, then uses the existing +/// read scope's single core-domain meter for selection, paths, frontier and output. +/// No cursor escapes. Caller visitor computation and copies retained by the caller +/// are outside owned instruction/retention quotas; logical bytes are not RSS. +public struct JSONTraversal: Sendable { + /// The immutable source to which every emitted absolute path belongs. + public let document: JSONDocument + /// Finite admission, work, path and logical-retention limits for each invocation. + public let limits: JSONLimits + /// Invocation-local rendezvous for qualification, absent from public consumers. + package var checkpointObserver: (@Sendable (UInt64) -> Void)? + + /// Captures a document without admission or scanning; each operation admits it. + /// - Parameters: + /// - document: An immutable canonical source, optionally with coherent mapping. + /// - limits: Finite cumulative limits, validated at operation entry. + public init(document: JSONDocument, limits: JSONLimits = .balanced) { + self.document = document; self.limits = limits + } + + /// Visits DFS preorder with object insertion order and ascending array indices. + /// - Parameters: + /// - scope: Strict absolute subtree selection, including scalar roots. + /// - options: Explicit depth, root-emission, kind and source-span policies. + /// - visitor: Synchronous caller work receiving metadata and the unchanged value. + /// - Returns: Coverage and all successful cumulative core statistics. + /// - Throws: Typed core access errors, invalid configuration, resource/caller errors or cancellation. + public func visit(scope: JSONTraversalScope = .subtree(at: []), + options: JSONTraversalOptions = .init(), + _ visitor: (JSONTraversalHit, JSON) throws -> JSONTraversalAction) + throws -> JSONOperationReport { + let report = try execute(scope: scope, options: options, collectionLimit: nil, visitor) + return .init(result: report.result.summary, statistics: report.statistics) + } + + /// Collects finite metadata through the same visitor engine and the same ledger. + /// - Parameters: + /// - scope: Strict absolute subtree selection. + /// - options: Explicit exploration and emission policies. + /// - maximumRecords: Nonnegative inclusive record bound; defaults to 100,000. + /// - Returns: Records, coverage and cumulative admission/traversal/output statistics. + /// - Throws: Collection-capacity refusal, typed access/configuration/resource errors or cancellation. + public func collect(scope: JSONTraversalScope = .subtree(at: []), + options: JSONTraversalOptions = .init(), maximumRecords: Int = 100_000) + throws -> JSONOperationReport { + try execute(scope: scope, options: options, collectionLimit: maximumRecords) { _, _ in .continue } + } + + private struct Frame { + let value: JSON + let path: JSONReadPath + var nextChild = 0 + var entered = false + } + + private struct Root { + let value: JSON + let path: JSONReadPath + } + + private func execute(scope selection: JSONTraversalScope, options: JSONTraversalOptions, + collectionLimit: Int?, + _ visitor: (JSONTraversalHit, JSON) throws -> JSONTraversalAction) + throws -> JSONOperationReport { + let scope = try JSONReadScope(document, limits: limits) + defer { scope.invalidate() } + let state = try scope.check() + state.meter.checkpointObserver = checkpointObserver + guard options.depthLimit.map({ $0 >= 0 }) ?? true, + collectionLimit.map({ $0 >= 0 }) ?? true else { + throw JSONFailure(.configuration, code: .invalidLimits) + } + try state.meter.retain(128) // fixed invocation summary/staging + let locations: [[JSONLocationComponent]], overlap: JSONScopeOverlapPolicy + try state.meter.retain(16) + switch selection { + case .subtree(let path): locations = [path]; overlap = .orderedOccurrences + case .virtual(let roots, let policy): locations = roots; overlap = policy + } + var roots: [Root] = [] + for location in locations { + try state.meter.work() + try state.meter.require(.pointerTokens, UInt64(location.count)) + var path = JSONReadPath(scope: scope, components: [], bytes: 0) + for part in location { + path = try scope.append(part, to: path.components, state: state) + _ = try scope.check() + } + let value = try scope.run(path.components) { try scope.select(path.components, state: $0) } + try state.meter.retain(64); roots.append(.init(value: value, path: path)) + } + if overlap == .rejectAmbiguous { + for second in roots.indices { + for first in 0.. 0 { + if action == .skipDescendants { + summary.prunedSubtrees += 1; frames[position].nextChild = childCount + summary.completion = .restricted + } else if options.depthLimit.map({ depth >= $0 }) ?? false { + summary.depthLimitedSubtrees += 1; frames[position].nextChild = childCount + summary.completion = .restricted + } + } + } + if frames[position].nextChild < childCount { + let ordinal = frames[position].nextChild + frames[position].nextChild += 1 + try state.meter.work() + let (component, child) = try Self.child(value, at: ordinal) + let path = try scope.append(component, to: frames[position].path.components, state: state) + try state.meter.retain(64) + frames.append(.init(value: child, path: path)) + } else { + frames.removeLast(); try state.meter.release(64) + } + } + // Refund live frontier only; consumed admission/path/visit work never resets. + while !frames.isEmpty { frames.removeLast(); try state.meter.release(64); try state.meter.work() } + if summary.completion == .visitorStopped { break } + } + while !seen.isEmpty { seen.removeLast(); try state.meter.release(16); try state.meter.work() } + while !roots.isEmpty { roots.removeLast(); try state.meter.release(64); try state.meter.work() } + try state.meter.release(144) + return try scope.complete(JSONTraversalCollection(nodes: records, summary: summary)) + } + + /// Exact metered prefix comparison; no String Set, value equality or unmetered hashing. + private static func path(_ lhs: [JSONLocationComponent], overlaps rhs: [JSONLocationComponent], + state: JSONCodingState) throws -> Bool { + for index in 0.. Int { + switch value { case .object(let o): return o.count; case .array(let a): return a.count; default: return 0 } + } + + private static func child(_ value: JSON, at index: Int) throws -> (JSONLocationComponent, JSON) { + switch value { + case .object(let object): return (.key(object[index].key), object[index].value) + case .array(let array): return (.index(index), array[index]) + default: throw JSONFailure(.configuration, code: .invalidOperation) + } + } + + private static func retainRecord(_ hit: JSONTraversalHit, state: JSONCodingState) throws { + try state.meter.retain(64) + for part in hit.absoluteLocation { + try state.meter.work(); try state.meter.retain(JSONLogicalWeights.indexSlot) + if case .key(let key) = part { + for _ in key.string.utf8 { try state.meter.work(); try state.meter.retain(1) } + } + } + } +} diff --git a/Sources/JSONNavigation/JSONTraversalTypes.swift b/Sources/JSONNavigation/JSONTraversalTypes.swift new file mode 100644 index 0000000..329ff20 --- /dev/null +++ b/Sources/JSONNavigation/JSONTraversalTypes.swift @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON + +/// A source value's existing JSON kind; absence is never a value kind. +public enum JSONTraversalKind: Sendable, Hashable, CaseIterable { + /// An ordered object. + case object + /// An array. + case array + /// An exact scalar-sequence string. + case string + /// An exact bounded decimal. + case number + /// A Boolean. + case boolean + /// Explicit JSON null. + case null + + internal init(_ value: JSON) { + switch value { + case .object: self = .object + case .array: self = .array + case .string: self = .string + case .number: self = .number + case .boolean: self = .boolean + case .null: self = .null + } + } +} + +/// An intentional synchronous visitor decision, never an operational-error substitute. +public enum JSONTraversalAction: Sendable { + /// Explore descendants normally. + case `continue` + /// Prune only this occurrence's descendants. + case skipDescendants + /// End this invocation with explicitly incomplete coverage. + case stop +} + +/// Explicit duplicate/ancestor treatment for an ordered selection of physical roots. +public enum JSONScopeOverlapPolicy: Sendable { + /// Reject repeated or ancestor/descendant root selections before any visitor call. + case rejectAmbiguous + /// Visit each selected subtree independently, including repeated occurrences. + case orderedOccurrences + /// Emit a physical typed path at most once; its first examined occurrence wins. + case uniquePhysicalNodes +} + +/// Existing physical subtrees in the captured canonical source, never a synthetic JSON array. +public enum JSONTraversalScope: Sendable { + /// A strict absolute typed path. An empty path selects the root, of any kind. + /// - Parameter at: Exact absolute key/index components. + case subtree(at: [JSONLocationComponent]) + /// Ordered roots, all strictly resolved before traversal. Empty selection is valid. + /// - Parameters: + /// - roots: Ordered absolute typed paths into the captured source. + /// - overlap: Explicit rejection, occurrence or exact-path deduplication policy. + case virtual(roots: [[JSONLocationComponent]], overlap: JSONScopeOverlapPolicy = .rejectAmbiguous) +} + +/// DFS emission and exploration policy. Admission always checks the entire captured source. +public struct JSONTraversalOptions: Sendable { + /// Maximum edges from a selected root, or nil for all admitted descendants. + public var depthLimit: Int? + /// Whether to emit each scope root; excluded roots are still visited. + public var includeScopeRoot: Bool + /// Kinds to emit, or nil for all six kinds. Filtering never prunes descendants. + public var emitKinds: Set? + /// Whether emitted hits include coherent original value spans, when available. + public var includeSourceSpans: Bool + + /// Configures deterministic insertion-order, depth-first preorder traversal. + /// - Parameters: + /// - depthLimit: Nonnegative edge bound; nil still uses finite core limits. + /// - includeScopeRoot: Emit roots as well as their descendants. + /// - emitKinds: Emission-only filter; an empty set emits no hits. + /// - includeSourceSpans: Request metered lookup of original value spans. + public init(depthLimit: Int? = nil, includeScopeRoot: Bool = true, + emitKinds: Set? = nil, includeSourceSpans: Bool = false) { + self.depthLimit = depthLimit; self.includeScopeRoot = includeScopeRoot + self.emitKinds = emitKinds; self.includeSourceSpans = includeSourceSpans + } +} + +/// Metadata for an existing node, without a retained subtree or an escaping cursor. +/// Paths address the traversal's captured document, not an arbitrary replacement root. +public struct JSONTraversalHit: Sendable, Equatable { + /// Absolute exact key/index components; [] is root and [.key("")] is not. + public let absoluteLocation: [JSONLocationComponent] + /// Zero-based root selection index; zero for a physical subtree. + public let scopeRootIndex: Int + /// Zero-based visited occurrence ordinal, including filtered nodes. + public let occurrenceIndex: UInt64 + /// Edges from this occurrence's selected scope root. + public let relativeDepth: Int + /// The unchanged source kind. + public let kind: JSONTraversalKind + /// An original value span only when requested from a coherent mapped document. + public let sourceSpan: JSONSourceSpan? +} + +/// Structural exploration coverage, independent of emission filters or source-map availability. +public enum JSONTraversalCompletion: Sendable { + /// All selected descendants were explored, even if filters emitted no hits. + case complete + /// A depth policy or visitor pruning omitted descendants. + case restricted + /// The visitor explicitly ended the walk; this never establishes global absence. + case visitorStopped +} + +/// Successful exploration facts. Resource failures and cancellation produce no summary. +public struct JSONTraversalSummary: Sendable { + /// Examined occurrences, including filtered nodes and explicitly repeated roots. + public internal(set) var visitedCount: UInt64 = 0 + /// Hits handed to the visitor or retained by collect. + public internal(set) var emittedCount: UInt64 = 0 + /// Maximum relative depth actually examined, zero for an empty selection. + public internal(set) var maximumDepthVisited: Int = 0 + /// Number of validated root selections. + public internal(set) var scopeRootCount: Int = 0 + /// Re-examined typed paths omitted under uniquePhysicalNodes; descendants are still explored. + public internal(set) var deduplicatedCount: UInt64 = 0 + /// Containers whose descendants were explicitly pruned by the visitor. + public internal(set) var prunedSubtrees: UInt64 = 0 + /// Containers whose descendants were omitted by the depth policy. + public internal(set) var depthLimitedSubtrees: UInt64 = 0 + /// Whether exploration was complete, policy-restricted or explicitly stopped. + public internal(set) var completion: JSONTraversalCompletion = .complete +} + +/// Bounded metadata records and their independent exploration coverage. +public struct JSONTraversalCollection: Sendable { + /// Source-ordered hits; no JSON subtrees or generated previews are retained. + public let nodes: [JSONTraversalHit] + /// Coverage and observed counts for the same invocation. + public let summary: JSONTraversalSummary +} + +/// A traversal-specific explicit capacity refusal, never a successful truncated result. +public enum JSONTraversalFailure: Error, Sendable, Equatable { + /// The next collected record would exceed the caller's finite inclusive bound. + /// - Parameter limit: The nonnegative inclusive maximum record count. + case collectionLimitExceeded(limit: Int) + /// Two validated virtual roots are identical or one is an ancestor of the other. + /// - Parameters: + /// - first: The earlier zero-based root selection index. + /// - second: The later zero-based root selection index. + case ambiguousRoots(first: Int, second: Int) +} diff --git a/Tests/JSONNavigationTests/Fixtures/Provenance.json b/Tests/JSONNavigationTests/Fixtures/Provenance.json new file mode 100644 index 0000000..cd3195e --- /dev/null +++ b/Tests/JSONNavigationTests/Fixtures/Provenance.json @@ -0,0 +1,19 @@ +{ + "origin": "Maintainer-provided Structural Intelligence S1 synthetic fixtures", + "packet_sha256": "d758f75d84271a97da515bf7b7ef70de7a61e91fdf949a511fe0fca2708156f8", + "adapted_algorithm": false, + "fixtures": [ + { + "path": "paths.json", + "sha256": "57ad0216911c5c581c6370500ea9564c6b0f906f4fb01c3e60d8789c4bf0ce26" + }, + { + "path": "structures.json", + "sha256": "756312755ad347361be8d7330e213c71c9c0e3faf4b4b1d9cdfa523cb0681945" + }, + { + "path": "source-revisions.json", + "sha256": "a28438f83a85c40ddcc69b174f517403ba6eb8dc94f3adf2bfadcd418878fe81" + } + ] +} diff --git a/Tests/JSONNavigationTests/Fixtures/paths.json b/Tests/JSONNavigationTests/Fixtures/paths.json new file mode 100644 index 0000000..8248119 --- /dev/null +++ b/Tests/JSONNavigationTests/Fixtures/paths.json @@ -0,0 +1,15 @@ +{ + "": "empty-key", + "0": "numeric-object-key", + "a/b": "slash-key", + "~": "tilde-key", + "a.b": "dotted-key", + "é": "composed", + "é": "decomposed", + "items": [ + "first-array-item", + { + "0": "nested-numeric-object-key" + } + ] +} diff --git a/Tests/JSONNavigationTests/Fixtures/source-revisions.json b/Tests/JSONNavigationTests/Fixtures/source-revisions.json new file mode 100644 index 0000000..8abe1ec --- /dev/null +++ b/Tests/JSONNavigationTests/Fixtures/source-revisions.json @@ -0,0 +1,18 @@ +{ + "rev1": { + "name": "sample", + "rows": [ + 1, + 2, + 3 + ] + }, + "rev2": { + "name": "sample", + "rows": [ + 1, + 2, + 4 + ] + } +} diff --git a/Tests/JSONNavigationTests/Fixtures/structures.json b/Tests/JSONNavigationTests/Fixtures/structures.json new file mode 100644 index 0000000..ed7cc0c --- /dev/null +++ b/Tests/JSONNavigationTests/Fixtures/structures.json @@ -0,0 +1,56 @@ +{ + "tree": { + "left": { + "leaf": 1 + }, + "right": [ + 2, + { + "leaf": 3 + } + ] + }, + "matrix": [ + [ + 0.1, + 9007199254740993 + ], + [ + -0.5, + 1.0 + ] + ], + "ragged": [ + [ + 1, + 2 + ], + [ + 3 + ] + ], + "graph": { + "nodes": [ + { + "id": "A" + }, + { + "id": "A" + } + ], + "edges": [ + { + "from": "A", + "to": "B" + } + ] + }, + "virtual": [ + { + "x": 1 + }, + { + "x": 1 + } + ] +} diff --git a/Tests/JSONNavigationTests/NavigationBoundaryTests.swift b/Tests/JSONNavigationTests/NavigationBoundaryTests.swift new file mode 100644 index 0000000..8c3b2ac --- /dev/null +++ b/Tests/JSONNavigationTests/NavigationBoundaryTests.swift @@ -0,0 +1,240 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Testing +import JSON +import JSONNavigation + +private func navigationFailure(_ body: () throws -> Void) -> JSONFailure? { + do { try body(); Issue.record("Expected a structured core failure"); return nil } + catch let error as JSONFailure { return error } + catch let error as JSONAccessFailure { return error.cause } + catch { Issue.record("Unexpected failure type: \(type(of: error))"); return nil } +} + +@Test func navigationAllScalarAndEmptyContainerRoots() throws { + for value: JSON in [.null, .boolean(true), .number(1), .string(""), .array([]), .object(try JSONObject())] { + let walk = JSONTraversal(document: .init(root: value)) + let report = try walk.collect() + #expect(report.result.nodes.count == 1 && report.result.nodes[0].absoluteLocation.isEmpty) + #expect(report.result.summary.completion == .complete) + #expect(try walk.collect(options: .init(includeScopeRoot: false)).result.nodes.isEmpty) + } +} + +@Test func navigationDepthZeroChildrenOnlyAndLeafCoverage() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":{"b":1},"c":2}"#))) + let zero = try walk.collect(options: .init(depthLimit: 0)) + #expect(zero.result.nodes.count == 1 && zero.result.summary.depthLimitedSubtrees == 1) + #expect(zero.result.summary.completion == .restricted) + let children = try walk.collect(options: .init(depthLimit: 1, includeScopeRoot: false)) + #expect(children.result.nodes.map(\.absoluteLocation) == [[.key("a")], [.key("c")]]) + let leaf = try walk.visit(scope: .subtree(at: [.key("c")]), options: .init(depthLimit: 0)) { _, _ in .skipDescendants } + #expect(leaf.result.completion == .complete && leaf.result.prunedSubtrees == 0) +} + +@Test func navigationStrictFailureCategoriesAndNoCallbacks() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":[null]}"#))) + let cases: [([JSONLocationComponent], JSONFailure.Category, JSONFailure.Code)] = [ + ([.key("missing")], .missingValue, .missingMember), + ([.index(0)], .typeMismatch, .wrongKind), + ([.key("a"), .key("0")], .typeMismatch, .wrongKind), + ([.key("a"), .index(-1)], .configuration, .invalidIndex), + ([.key("a"), .index(1)], .missingValue, .missingTarget), + ([.key("a"), .index(0), .key("x")], .typeMismatch, .wrongKind) + ] + for (path, category, code) in cases { + var calls = 0 + let failure = navigationFailure { _ = try walk.visit(scope: .subtree(at: path)) { _, _ in calls += 1; return .continue } } + #expect(failure?.category == category && failure?.code == code && calls == 0) + } +} + +@Test func navigationEmptyEmissionIsCompleteAndNotMissing() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":[1,null]}"#))) + let report = try walk.collect(options: .init(emitKinds: []), maximumRecords: 0) + #expect(report.result.nodes.isEmpty && report.result.summary.visitedCount == 4) + #expect(report.result.summary.completion == .complete) + #expect(navigationFailure { _ = try walk.collect(scope: .subtree(at: [.key("missing")]), options: .init(emitKinds: [])) }?.category == .missingValue) +} + +@Test func navigationInsertionOrderIsIndependentOfValueEquality() throws { + let first = try JSON.parse(#"{"z":1,"a":2}"#), second = try JSON.parse(#"{"a":2,"z":1}"#) + #expect(first == second) + let a = try JSONTraversal(document: .init(root: first)).collect().result.nodes.map(\.absoluteLocation) + let b = try JSONTraversal(document: .init(root: second)).collect().result.nodes.map(\.absoluteLocation) + #expect(a == [[], [.key("z")], [.key("a")]]) + #expect(b == [[], [.key("a")], [.key("z")]]) +} + +@Test func navigationPreservesExactDecimalsAndText() throws { + let root = try JSON.parse(#"[9007199254740993,0.1,1,1.0,1e0,"é","e\u0301"]"#) + var values: [JSON] = [] + _ = try JSONTraversal(document: .init(root: root)).visit(options: .init(includeScopeRoot: false)) { _, value in values.append(value); return .continue } + #expect(values[0].numberValue == (try JSONNumber(parsing: "9007199254740993"))) + #expect(values[1].numberValue == (try JSONNumber(parsing: "0.1"))) + #expect(values[2] == values[3] && values[3] == values[4]) + #expect(values[3].numberValue?.originalLexeme == "1.0") + #expect(values[4].numberValue?.originalLexeme == "1e0") + #expect(values[5] != values[6]) +} + +@Test func navigationSubtreeMetamorphismAndDeterminism() throws { + let root = try JSON.parse(#"{"x":[{"":"a","a/b":[]},{"~":{},".[]":0}],"y":1}"#) + let walk = JSONTraversal(document: .init(root: root)) + let all = try walk.collect().result.nodes + for path: [JSONLocationComponent] in [[], [.key("x")], [.key("x"), .index(0)], [.key("x"), .index(1), .key("~")]] { + let subset = try walk.collect(scope: .subtree(at: path)).result.nodes + let expected = all.filter { $0.absoluteLocation.starts(with: path) }.map(\.absoluteLocation) + #expect(subset.map(\.absoluteLocation) == expected) + #expect(subset.map(\.relativeDepth) == subset.map { $0.absoluteLocation.count - path.count }) + #expect(try walk.collect(scope: .subtree(at: path)).result.nodes == subset) + } +} + +@Test func navigationMappedAndReplacedSourceCoherence() throws { + let source = try JSONParser().parseDocument(Data(#"{"a/b": [1.0, null], "é":2,"e\u0301":3}"#.utf8), source: "synthetic") + let original = JSONTraversal(document: source) + let mapped = try original.collect(options: .init(includeSourceSpans: true)) + for hit in mapped.result.nodes { + #expect(hit.sourceSpan == (try source.sourceMap?.span(at: hit.absoluteLocation))) + #expect(hit.sourceSpan != nil) + } + let rewritten = source.replacingRoot(.array([.null])) + let new = try JSONTraversal(document: rewritten).collect(options: .init(includeSourceSpans: true)) + #expect(new.result.nodes.allSatisfy { $0.sourceSpan == nil }) + #expect(original.document.root == source.root && original.document.sourceBytes != nil) + #expect(rewritten.sourceMap == nil && rewritten.sourceBytes == nil && rewritten.source == nil) + #expect(try original.collect(options: .init(includeSourceSpans: true)).result.nodes == mapped.result.nodes) +} + +@Test func navigationDeepIterativeWalkAndFullAdmission() throws { + var value = JSON.null + for _ in 0..<200 { value = .array([value]) } + var limits = JSONLimits.balanced; limits[.maxDepth] = 256; limits[.pointerTokens] = 256 + let report = try JSONTraversal(document: .init(root: value), limits: limits).collect() + #expect(report.result.nodes.count == 201 && report.result.summary.maximumDepthVisited == 200) + let failure = navigationFailure { _ = try JSONTraversal(document: .init(root: value)).collect(options: .init(depthLimit: 0)) } + #expect(failure?.resource?.metric == .maxDepth) +} + +@Test func navigationWideObjectDoesNotSortOrLoseKeys() throws { + let members = (0..<1000).map { JSONMember(JSONKey("k\(999 - $0)"), value: .number(JSONNumber(integer: $0))) } + let root = JSON.object(try JSONObject(members: members)) + let report = try JSONTraversal(document: .init(root: root)).collect() + #expect(report.result.nodes.dropFirst().map(\.absoluteLocation) == members.map { [.key($0.key)] }) + #expect(report.result.summary.visitedCount == 1001 && report.statistics[.nodes] == 1001) +} + +@Test func navigationCallerErrorsPropagateAndCopiesAreCallerOwned() throws { + enum Caller: Error { case witness } + let walk = JSONTraversal(document: .init(root: .array([1,2]))) + #expect(throws: Caller.witness) { try walk.visit { _, _ in throw Caller.witness } } + var retained: [JSONTraversalHit] = [] + let report = try walk.visit { hit, _ in retained.append(hit); return .continue } + #expect(retained.count == 3 && report.statistics[.retainedBytes] == 0) +} + +@Test func navigationCollectionBoundaryAndInvalidPolicies() throws { + let walk = JSONTraversal(document: .init(root: .array([1,2]))) + #expect(try walk.collect(maximumRecords: 3).result.nodes.count == 3) + #expect(throws: JSONTraversalFailure.collectionLimitExceeded(limit: 2)) { try walk.collect(maximumRecords: 2) } + #expect(throws: JSONTraversalFailure.collectionLimitExceeded(limit: 0)) { try walk.collect(maximumRecords: 0) } + #expect(navigationFailure { _ = try walk.collect(maximumRecords: -1) }?.category == .configuration) + #expect(navigationFailure { _ = try walk.collect(options: .init(depthLimit: -1)) }?.category == .configuration) +} + +@Test func navigationRetainedOutputAndInclusivePeak() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"long/key":[1,2]}"#))) + let report = try walk.collect() + let expected: UInt64 = report.result.nodes.reduce(0) { total, hit in + total + 64 + hit.absoluteLocation.reduce(UInt64(0)) { n, part in + switch part { case .key(let key): return n + 32 + UInt64(key.string.utf8.count); case .index: return n + 32 } + } + } + #expect(report.statistics[.retainedBytes] == expected) + var limits = JSONLimits.balanced; limits[.retainedBytes] = report.statistics.peakLogicalRetainedBytes + #expect(try JSONTraversal(document: walk.document, limits: limits).collect().statistics.peakLogicalRetainedBytes == limits[.retainedBytes]) + limits[.retainedBytes] -= 1 + #expect(navigationFailure { _ = try JSONTraversal(document: walk.document, limits: limits).collect() }?.resource?.metric == .retainedBytes) +} + +@Test func navigationCorePathNodeAndWorkBounds() throws { + let document = JSONDocument(root: .array([.null])) + var limits = JSONLimits.balanced; limits[.pointerTokens] = 0 + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect() }?.resource?.metric == .pointerTokens) + #expect(try JSONTraversal(document: document, limits: limits).collect(options: .init(depthLimit: 0)).result.nodes.count == 1) + limits = .balanced; limits[.pointerBytes] = 1 + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect() }?.resource?.metric == .pointerBytes) + limits = .balanced; limits[.nodes] = 1 + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect(options: .init(depthLimit: 0)) }?.resource?.metric == .nodes) + limits = .balanced; limits[.workUnits] = 0 + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect() }?.resource?.metric == .workUnits) +} + +@Test func navigationCancellationBeforeAdmission() async { + let task = Task { + withUnsafeCurrentTask { $0?.cancel() } + return try JSONTraversal(document: .init(root: .null)).collect() + } + do { _ = try await task.value; Issue.record("Cancelled admission succeeded") } + catch { #expect(error is CancellationError) } +} + +@Test func navigationCancellationAfterVisitorCannotBecomeStop() async { + let task = Task { + try JSONTraversal(document: .init(root: .array([1,2]))).visit { _, _ in + withUnsafeCurrentTask { $0?.cancel() } + return .stop + } + } + do { _ = try await task.value; Issue.record("Cancellation became successful stop") } + catch { #expect(error is CancellationError) } +} + +@Test func navigationCancellationInsideOwnedFilteredLoop() async throws { + let document = JSONDocument(root: .array(Array(repeating: .null, count: 1000))) + let admitted = try document.readWithReport { _ in () }.statistics[.workUnits] + let task = Task { + var walk = JSONTraversal(document: document) + walk.checkpointObserver = { work in + if work >= admitted + 100 { withUnsafeCurrentTask { $0?.cancel() } } + } + return try walk.collect(options: .init(emitKinds: [])) + } + do { _ = try await task.value; Issue.record("Owned filtered loop ignored cancellation") } + catch { #expect(error is CancellationError) } +} + +@Test func navigationSharedImmutableRootIndependentInvocations() async throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"é":[1,2],"e\u0301":[3]}"#))) + let expected = try walk.collect().result.nodes + try await withThrowingTaskGroup(of: Void.self) { group in + for _ in 0..<12 { + group.addTask { () async throws -> Void in + let actual = try walk.collect().result.nodes + #expect(actual == expected) + } + } + try await group.waitForAll() + } +} + +@Test func navigationDocumentSourceAndSpanWorkAreCumulative() throws { + let document = try JSONParser().parseDocument(Data(#"{"a":[1,2]}"#.utf8)) + let basic = try JSONTraversal(document: document).collect() + let mapped = try JSONTraversal(document: document).collect(options: .init(includeSourceSpans: true)) + #expect(mapped.statistics[.workUnits] > basic.statistics[.workUnits]) + var limits = JSONLimits.balanced; limits[.inputBytes] = UInt64(document.sourceBytes!.count) + #expect(try JSONTraversal(document: document, limits: limits).collect().result.nodes.count == 4) + limits[.inputBytes] -= 1 + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect() }?.resource?.metric == .inputBytes) + limits = .balanced; limits[.workUnits] = basic.statistics[.workUnits] + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect(options: .init(includeSourceSpans: true)) }?.resource?.metric == .workUnits) +} + +@Test func navigationFailedInvocationDoesNotPoisonFreshWork() throws { + let document = JSONDocument(root: .array([1,2])) + var limits = JSONLimits.balanced; limits[.workUnits] = 1 + #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect() }?.category == .resource) + #expect(try JSONTraversal(document: document).collect().result.summary.completion == .complete) +} diff --git a/Tests/JSONNavigationTests/PacketWitnessTests.swift b/Tests/JSONNavigationTests/PacketWitnessTests.swift new file mode 100644 index 0000000..4dd4afe --- /dev/null +++ b/Tests/JSONNavigationTests/PacketWitnessTests.swift @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Testing +import JSON +import JSONNavigation + +private func navigationFixture(_ name: String) throws -> JSON { + let folder = URL(fileURLWithPath: #filePath).deletingLastPathComponent().appendingPathComponent("Fixtures") + return try JSONParser().parse(Data(contentsOf: folder.appendingPathComponent(name))) +} + +@Test func navigationPinnedPacketPathsAndUnevenDFS() throws { + let root = try navigationFixture("paths.json") + var witnessed = 0 + _ = try JSONTraversal(document: .init(root: root)).visit { hit, value in + let tokens = hit.absoluteLocation.map { p in + switch p { case .key(let k): return k.string; case .index(let i): return String(i) } + } + let resolved = try JSONPointer(tokens: tokens).resolve(in: root) + #expect(resolved == value) + witnessed += 1 + return .continue + } + #expect(witnessed == 12) + let tree = try navigationFixture("structures.json") + let paths = try JSONTraversal(document: .init(root: tree)).collect(scope: .subtree(at: [.key("tree")])).result.nodes.map(\.absoluteLocation) + #expect(paths == [[.key("tree")], [.key("tree"), .key("left")], [.key("tree"), .key("left"), .key("leaf")], [.key("tree"), .key("right")], [.key("tree"), .key("right"), .index(0)], [.key("tree"), .key("right"), .index(1)], [.key("tree"), .key("right"), .index(1), .key("leaf")]]) +} + +@Test func navigationCapturedRevisionNeverRebinds() throws { + let revisions = try navigationFixture("source-revisions.json") + let old = try revisions.requiredMember("rev1"), new = try revisions.requiredMember("rev2") + let oldTraversal = JSONTraversal(document: .init(root: old)) + let newTraversal = JSONTraversal(document: oldTraversal.document.replacingRoot(new)) + var oldTail = JSON.null, newTail = JSON.null + _ = try oldTraversal.visit { hit, value in if hit.absoluteLocation == [.key("rows"), .index(2)] { oldTail = value }; return .continue } + _ = try newTraversal.visit { hit, value in if hit.absoluteLocation == [.key("rows"), .index(2)] { newTail = value }; return .continue } + #expect(oldTail == 3 && newTail == 4 && oldTraversal.document.root != newTraversal.document.root) +} diff --git a/Tests/JSONNavigationTests/PhysicalTraversalTests.swift b/Tests/JSONNavigationTests/PhysicalTraversalTests.swift new file mode 100644 index 0000000..fc48974 --- /dev/null +++ b/Tests/JSONNavigationTests/PhysicalTraversalTests.swift @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +import JSON +import JSONNavigation + +@Test func navigationPhysicalPreorderAndTypedAuthority() throws { + let root = try JSON.parse(#"{"":"empty","0":"key","items":[{"a/b":9007199254740993},null],"é":1,"e\u0301":2}"#) + let walk = JSONTraversal(document: .init(root: root)) + var paths: [[JSONLocationComponent]] = [] + let report = try walk.visit { hit, value in + paths.append(hit.absoluteLocation) + let tokens = hit.absoluteLocation.map { part in + switch part { case .key(let k): return k.string; case .index(let i): return String(i) } + } + #expect(try JSONPointer(tokens: tokens).resolve(in: root) == value) + let read = try root.read { cursor in + var selected = cursor + for part in hit.absoluteLocation { + switch part { case .key(let k): selected = try selected[k]; case .index(let i): selected = try selected[i] } + } + return try selected.requiredValue() + } + #expect(read == value) + return .continue + } + #expect(paths == [[], [.key("")], [.key("0")], [.key("items")], [.key("items"), .index(0)], [.key("items"), .index(0), .key("a/b")], [.key("items"), .index(1)], [.key("é")], [.key("e\u{301}")]]) + #expect(report.result.completion == .complete) + #expect(report.result.visitedCount == 9) + #expect(report.statistics[.nodes] == 9) // exactly one root admission + #expect(report.statistics[.retainedBytes] == 0) +} + +@Test func navigationPhysicalScalarAndStrictFailures() throws { + let scalar = JSONTraversal(document: .init(root: .null)) + #expect(try scalar.collect().result.nodes.count == 1) + #expect(try scalar.collect(options: .init(includeScopeRoot: false)).result.nodes.isEmpty) + let root = try JSON.parse(#"{"x":null,"a":[1]}"#) + let walk = JSONTraversal(document: .init(root: root)) + #expect(try walk.collect(scope: .subtree(at: [.key("x")])).result.nodes.first?.kind == .null) + for path: [JSONLocationComponent] in [[.key("missing")], [.key("a"), .key("0")], [.key("a"), .index(-1)], [.key("a"), .index(1)], [.key("x"), .key("z")]] { + #expect(throws: (any Error).self) { try walk.collect(scope: .subtree(at: path)) } + } +} + +@Test func navigationPhysicalPoliciesDoNotConflateCoverage() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":[1,2],"b":3}"#))) + let filtered = try walk.collect(options: .init(emitKinds: [.number])) + #expect(filtered.result.summary.visitedCount == 5 && filtered.result.nodes.count == 3) + #expect(filtered.result.summary.completion == .complete) + let shallow = try walk.collect(options: .init(depthLimit: 1, includeScopeRoot: false)) + #expect(shallow.result.nodes.count == 2 && shallow.result.summary.completion == .restricted) + let stopped = try walk.visit { _, _ in .stop } + #expect(stopped.result.completion == .visitorStopped && stopped.result.visitedCount == 1) + let pruned = try walk.visit { hit, _ in hit.absoluteLocation == [.key("a")] ? .skipDescendants : .continue } + #expect(pruned.result.visitedCount == 3 && pruned.result.prunedSubtrees == 1) +} + +@Test func navigationOneLedgerAdmissionSelectionAndCollection() throws { + let root = try JSON.parse(#"{"a":[1,2]}"#) + let full = try JSONTraversal(document: .init(root: root)).collect(scope: .subtree(at: [.key("a")])) + let used = full.statistics[.workUnits] + var exact = JSONLimits.balanced; exact[.workUnits] = used + #expect(try JSONTraversal(document: .init(root: root), limits: exact).collect(scope: .subtree(at: [.key("a")])).statistics[.workUnits] == used) + exact[.workUnits] = used - 1 + #expect(throws: JSONFailure.self) { try JSONTraversal(document: .init(root: root), limits: exact).collect(scope: .subtree(at: [.key("a")])) } + #expect(throws: JSONTraversalFailure.collectionLimitExceeded(limit: 2)) { try JSONTraversal(document: .init(root: root)).collect(maximumRecords: 2) } + #expect(full.statistics[.nodes] == 4 && full.statistics[.retainedBytes] > 0) +} diff --git a/Tests/JSONNavigationTests/VirtualScopeTests.swift b/Tests/JSONNavigationTests/VirtualScopeTests.swift new file mode 100644 index 0000000..fc5addd --- /dev/null +++ b/Tests/JSONNavigationTests/VirtualScopeTests.swift @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +import JSON +import JSONNavigation + +@Test func navigationVirtualRejectsAmbiguityBeforeCallbacks() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":{"b":1},"c":1}"#))) + for roots: [[JSONLocationComponent]] in [[[.key("a")], [.key("a")]], [[.key("a")], [.key("a"), .key("b")]], [[], [.key("c")]]] { + var calls = 0 + #expect(throws: JSONTraversalFailure.ambiguousRoots(first: 0, second: 1)) { + try walk.visit(scope: .virtual(roots: roots)) { _, _ in calls += 1; return .continue } + } + #expect(calls == 0) + } +} + +@Test func navigationVirtualOrderedRepeatsRetainOriginAndDepth() throws { + let document = JSONDocument(root: try JSON.parse(#"{"a":{"b":1},"c":1}"#)) + let report = try JSONTraversal(document: document).collect(scope: .virtual(roots: [[.key("a")], [.key("a"), .key("b")], [.key("a")]], overlap: .orderedOccurrences)) + #expect(report.result.nodes.map(\.scopeRootIndex) == [0,0,1,2,2]) + #expect(report.result.nodes.map(\.relativeDepth) == [0,1,0,0,1]) + #expect(report.result.nodes.map(\.occurrenceIndex) == [0,1,2,3,4]) + #expect(report.result.summary.visitedCount == 5 && report.result.summary.deduplicatedCount == 0) + #expect(report.statistics[.nodes] == 4) // input admission is not restarted for each root +} + +@Test func navigationVirtualUniqueUsesPathsRatherThanEqualValues() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":1,"b":1,"é":1,"e\u0301":1}"#))) + let report = try walk.collect(scope: .virtual(roots: [[.key("a")], [], [.key("a")]], overlap: .uniquePhysicalNodes)) + #expect(report.result.nodes.map(\.absoluteLocation) == [[.key("a")], [], [.key("b")], [.key("é")], [.key("e\u{301}")]]) + #expect(report.result.nodes.map(\.scopeRootIndex) == [0,1,1,1,1]) + #expect(report.result.summary.deduplicatedCount == 2 && report.result.summary.visitedCount == 7) + #expect(report.result.summary.completion == .complete) +} + +@Test func navigationVirtualExactUnicodeAndSiblingRootsAreUnambiguous() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"é":1,"e\u0301":2,"0":3,"a":[4]}"#))) + let report = try walk.collect(scope: .virtual(roots: [[.key("é")], [.key("e\u{301}")], [.key("0")], [.key("a"), .index(0)]])) + #expect(report.result.nodes.count == 4 && report.result.summary.completion == .complete) +} + +@Test func navigationVirtualValidatesEveryRootBeforeAnyCallback() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":1}"#))) + for policy: JSONScopeOverlapPolicy in [.rejectAmbiguous, .orderedOccurrences, .uniquePhysicalNodes] { + var calls = 0 + do { + _ = try walk.visit(scope: .virtual(roots: [[.key("a")], [.key("missing")]], overlap: policy)) { _, _ in calls += 1; return .continue } + Issue.record("Missing virtual root accepted") + } catch let error as JSONAccessFailure { #expect(error.cause.category == .missingValue) } + #expect(calls == 0) + } +} + +@Test func navigationVirtualEmptySelectionIsCompleteAfterAdmission() throws { + let report = try JSONTraversal(document: .init(root: .array([1,2]))).collect(scope: .virtual(roots: []), maximumRecords: 0) + #expect(report.result.nodes.isEmpty && report.result.summary.scopeRootCount == 0) + #expect(report.result.summary.completion == .complete && report.statistics[.nodes] == 3) +} + +@Test func navigationVirtualUniqueDoesNotPruneRevisitedFrontier() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":{"b":{"c":1}}}"#))) + let report = try walk.collect(scope: .virtual(roots: [[], [.key("a")]], overlap: .uniquePhysicalNodes), options: .init(depthLimit: 2)) + #expect(report.result.nodes.map(\.absoluteLocation) == [[], [.key("a")], [.key("a"), .key("b")], [.key("a"), .key("b"), .key("c")]]) + #expect(report.result.nodes.last?.relativeDepth == 2 && report.result.nodes.last?.scopeRootIndex == 1) + #expect(report.result.summary.depthLimitedSubtrees == 1 && report.result.summary.completion == .restricted) +} + +@Test func navigationVirtualComparisonAndCollectionShareExactBudget() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":[1,2],"b":[3,4]}"#))) + let selection = JSONTraversalScope.virtual(roots: [[.key("a")], [.key("b")]]) + let baseline = try walk.collect(scope: selection) + var limits = JSONLimits.balanced; limits[.workUnits] = baseline.statistics[.workUnits] + #expect(try JSONTraversal(document: walk.document, limits: limits).collect(scope: selection).statistics == baseline.statistics) + limits[.workUnits] -= 1 + do { _ = try JSONTraversal(document: walk.document, limits: limits).collect(scope: selection); Issue.record("Work quota reset") } + catch let error as JSONFailure { #expect(error.resource?.metric == .workUnits) } + #expect(try walk.collect(scope: selection).result.nodes == baseline.result.nodes) +} + +@Test func navigationVirtualUniqueFilteredRootFirstOccurrenceWins() throws { + let walk = JSONTraversal(document: .init(root: try JSON.parse(#"{"a":1}"#))) + let report = try walk.collect(scope: .virtual(roots: [[.key("a")], []], overlap: .uniquePhysicalNodes), options: .init(includeScopeRoot: false)) + #expect(report.result.nodes.isEmpty && report.result.summary.deduplicatedCount == 1) + #expect(report.result.summary.completion == .complete) +} + +@Test func navigationVirtualOverlapComparisonCannotHideWork() throws { + let keys = (0..<60).map { JSONKey(String(repeating: "x", count: 120) + String($0)) } + let members = keys.map { JSONMember($0, value: .null) } + let document = JSONDocument(root: .object(try JSONObject(members: members))) + let roots = keys.map { [JSONLocationComponent.key($0)] } + let options = JSONTraversalOptions(depthLimit: 0, emitKinds: []) + let baseline = try JSONTraversal(document: document).collect(scope: .virtual(roots: roots, overlap: .orderedOccurrences), options: options) + var limits = JSONLimits.balanced; limits[.workUnits] = baseline.statistics[.workUnits] + do { + _ = try JSONTraversal(document: document, limits: limits).collect(scope: .virtual(roots: roots), options: options) + Issue.record("Pairwise exact comparisons escaped the cumulative quota") + } catch let error as JSONFailure { #expect(error.resource?.metric == .workUnits) } +} diff --git a/Tools/NavigationQualification/Consumer.swift b/Tools/NavigationQualification/Consumer.swift new file mode 100644 index 0000000..0859268 --- /dev/null +++ b/Tools/NavigationQualification/Consumer.swift @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: Apache-2.0 +// Compiled as an external module by Scripts/qualify_navigation.py; no private imports. +import Foundation +import JSON +import JSONNavigation + +@main struct NavigationConsumer { + static func main() throws { + var checks = 0 + func check(_ id: String, _ condition: Bool) { + precondition(condition, id); checks += 1 + print("NAVIGATION_CHECK \(id) passed") + } + let text = #"{"":"empty","0":"key","items":[{"n":9007199254740993},null],"é":1,"e\u0301":2,"null":null}"# + let root = try JSONParser().parse(text) + // BEFORE: the core continues to own one-node lookup and native projection. + let whole = try JSONPointer(parsing: "").resolve(in: root) + let empty = try JSONPointer(parsing: "/").resolve(in: root) + check("root-empty-key", whole == root && empty == .string("empty")) + let keyPath: [JSONLocationComponent] = [.key("0")] + let indexPath: [JSONLocationComponent] = [.index(0)] + check("typed-key-index", keyPath != indexPath) + let presence = try root.read { cursor in + (try cursor.presence(of: "null"), try cursor.presence(of: "absent")) + } + check("presence", presence.0 == .null && presence.1 == .absent) + check("scalar-keys", root.objectValue!.count == 6) + let count = try root.read { try $0["items"][0]["n"].requiredInteger(as: UInt64.self) } + check("native-exact", count == 9_007_199_254_740_993) + + // AFTER: one many-node operation, exact metadata and unchanged source values. + let traversal = JSONTraversal(document: JSONDocument(root: root)) + var parity = true + let visit = try traversal.visit { hit, value in + let tokens = hit.absoluteLocation.map { part in + switch part { case .key(let key): return key.string; case .index(let i): return String(i) } + } + let selected = try JSONPointer(tokens: tokens).resolve(in: root) + parity = parity && selected == value + return .continue + } + check("pointer-parity", parity && visit.result.visitedCount == 10) + let collected = try traversal.collect() + check("metadata-collection", collected.result.nodes.count == 10 && collected.result.summary.completion == .complete) + let path: [JSONLocationComponent] = [.key("items"), .index(0)] + let repeated = try traversal.collect(scope: .virtual(roots: [path,path], overlap: .orderedOccurrences)) + check("ordered-occurrences", repeated.result.nodes.map(\.scopeRootIndex) == [0,0,1,1]) + let unique = try traversal.collect(scope: .virtual(roots: [path,path], overlap: .uniquePhysicalNodes)) + check("unique-paths", unique.result.nodes.count == 2 && unique.result.summary.deduplicatedCount == 2) + var ambiguous = false + do { _ = try traversal.collect(scope: .virtual(roots: [path,path])) } + catch JSONTraversalFailure.ambiguousRoots(first: 0, second: 1) { ambiguous = true } + check("default-overlap-rejection", ambiguous) + var missing = false + do { _ = try traversal.collect(scope: .subtree(at: [.key("missing")])) } + catch let error as JSONAccessFailure { missing = error.cause.category == .missingValue } + check("typed-missing", missing) + let filtered = try traversal.collect(options: .init(emitKinds: []), maximumRecords: 0) + check("complete-empty-filter", filtered.result.nodes.isEmpty && filtered.result.summary.completion == .complete && filtered.result.summary.visitedCount == 10) + var capacity = false + do { _ = try traversal.collect(maximumRecords: 0) } + catch JSONTraversalFailure.collectionLimitExceeded(limit: 0) { capacity = true } + check("collection-limit", capacity) + check("cumulative-report", collected.statistics[.nodes] == 10 && collected.statistics[.workUnits] > 0 && collected.statistics.domain == .core) + let document = try JSONParser().parseDocument(Data(text.utf8)) + let spans = try JSONTraversal(document: document).collect(options: .init(includeSourceSpans: true)) + check("coherent-spans", spans.result.nodes.allSatisfy { $0.sourceSpan != nil }) + let replaced = try JSONTraversal(document: document.replacingRoot(.null)).collect(options: .init(includeSourceSpans: true)) + check("replacement-invalidates-spans", replaced.result.nodes[0].sourceSpan == nil) + let stop = try traversal.visit { _, _ in .stop } + check("intentional-stop", stop.result.completion == .visitorStopped && stop.result.visitedCount == 1) + var limits = JSONLimits.balanced; limits[.workUnits] = collected.statistics[.workUnits] - 1 + var quota = false + do { _ = try JSONTraversal(document: traversal.document, limits: limits).collect() } + catch let error as JSONFailure { quota = error.resource?.metric == .workUnits } + check("cumulative-quota", quota) + print("NAVIGATION_RESOURCE work=\(collected.statistics[.workUnits]) retained=\(collected.statistics[.retainedBytes]) peak=\(collected.statistics.peakLogicalRetainedBytes) nodes=\(collected.statistics[.nodes])") + print("NAVIGATION_CONSUMER passed cases=\(checks)") + } +} From f168e9dc602824ce467126d17be26e99027a1625 Mon Sep 17 00:00:00 2001 From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com> Date: Thu, 15 Oct 2026 23:56:43 +0200 Subject: [PATCH 2/6] Preserve core key admission in integrated scope queries Replay 6e5e5fa from PR #34; generated source identity refreshed for composition-enabled integration. --- .../StructuralIntelligence/API-DECISIONS.md | 4 ++-- .../StructuralIntelligence/Progress.json | 2 +- .../StructuralIntelligence/SOURCE-AUTHORITY.md | 1 + Documentation/CI/README.md | 6 +++--- Documentation/Qualification/TestDiscovery.json | 6 +++--- Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +- Sources/JSONNavigation/JSONTraversal.swift | 5 +++++ .../NavigationBoundaryTests.swift | 14 ++++++++++++++ 8 files changed, 30 insertions(+), 10 deletions(-) diff --git a/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md b/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md index e45407f..ade93f7 100644 --- a/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md +++ b/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md @@ -87,10 +87,10 @@ owned-loop cancellation qualification; no public consumer can install it. ## Qualification and promotion boundary -The reviewed new inventory is 36 nonparameterized Navigation tests, including the +The reviewed new inventory is 37 nonparameterized Navigation tests, including the three pinned synthetic packet documents. Existing function identities and all 325 native/Linux parameter invocations remain; WASI retains its separate 288. Full roots -become 582 native, 566 Linux and 478 WASI, and shared-state counts 42/41. All 126 +become 583 native, 567 Linux and 479 WASI, and shared-state counts 42/41. All 126 maintenance identities, corpus policies and 49 cap probes remain required. `Scripts/ci.py --lane navigation` checks exact focused discovery, Debug/Release, diff --git a/Documentation/Architecture/StructuralIntelligence/Progress.json b/Documentation/Architecture/StructuralIntelligence/Progress.json index 384e879..5984415 100644 --- a/Documentation/Architecture/StructuralIntelligence/Progress.json +++ b/Documentation/Architecture/StructuralIntelligence/Progress.json @@ -4,7 +4,7 @@ "implementation_base": "037e735dc6224789e4d04d2c18d15b8d27440198", "packet_sha256": "d758f75d84271a97da515bf7b7ef70de7a61e91fdf949a511fe0fca2708156f8", "N0": "source authorities reconciled; core-only seven-witness consumer and package-scoped one-ledger composition compiled/executed", - "N1": "experimental physical/virtual DFS, bounded collection and 36 focused behavior functions implemented; complete platform qualification in progress", + "N1": "experimental physical/virtual DFS, bounded collection and 37 focused behavior functions implemented; complete platform qualification in progress", "public_product": false, "public_products_preserved": 7, "core_semantic_changes": false, diff --git a/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md b/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md index 4ae2b3c..de008a4 100644 --- a/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md +++ b/Documentation/Architecture/StructuralIntelligence/SOURCE-AUTHORITY.md @@ -26,6 +26,7 @@ phases use the **same** read-scope state and its noncopyable core-domain meter. | Admission and source lifetime | [JSONReadScope.init/check/complete/invalidate](../../../Sources/JSON/Reading/JSONReadScope.swift) | Exactly one root/source admission and terminal ledger; complete releases scope ownership | | Exact keys / insertion order | [JSONObject](../../../Sources/JSON/Value/JSONObject.swift), JSONKey | Visit ordinal members; no String sort, dictionary conversion or duplicate index | | Typed physical selection | JSONReadScope.select inside run | Strict key/index parent checks and existing JSONAccessFailure; no Pointer-token reinterpretation | +| Requested scope-key admission | JSONAdmission.text(key: true), as in JSONReadCursor's JSONKey subscript | Caller query keys consume the same cumulative key/text capacities before path staging or lookup | | Typed path staging | JSONReadScope.append and JSONReadPath | Existing pointerTokens/pointerBytes/diagnosticBytes checks, path copy/UTF-8 work and live refunds | | Meter/operational failure | [JSONWorkMeter](../../../Sources/JSON/Limits/JSONWorkMeter.swift), JSONCodingState | Shared cumulative work, capacity/retention, cancellation and terminal health; never reset | | DFS child enumeration | JSON array indices / JSONObject ordinal collection access | New iterative frontier; each node/edge/frame/callback handoff checked and charged | diff --git a/Documentation/CI/README.md b/Documentation/CI/README.md index 2e50aeb..2b5c044 100644 --- a/Documentation/CI/README.md +++ b/Documentation/CI/README.md @@ -2,10 +2,10 @@ ## Experimental N1 branch -The Structural Intelligence branch adds exactly 36 Navigation functions and one +The Structural Intelligence branch adds exactly 37 Navigation functions and one shared-root safety function, keeping every existing identity/case. Root expectations -are 582 native/566 Linux with 325 parameter invocations; TSan is 42/41. WASI's -separate core scope is 478 functions/288 parameters. No new public product is exposed. +are 583 native/567 Linux with 325 parameter invocations; TSan is 42/41. WASI's +separate core scope is 479 functions/288 parameters. No new public product is exposed. `--lane navigation` checks focused discovery, Debug/Release, explicit strict-memory compilation and the external compiler/consumer/experimental DocC audit. Native/Linux include that consumer alongside their existing gates; all 49 core cap outcomes and diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json index 805b48a..1071562 100644 --- a/Documentation/Qualification/TestDiscovery.json +++ b/Documentation/Qualification/TestDiscovery.json @@ -10,7 +10,7 @@ "JSONHTMLTests": 97, "JSONBridgesTests": 10, "JSONCLITests": 88, - "JSONNavigationTests": 36 + "JSONNavigationTests": 37 }, "parameterized_invocations": 325, "composition_external_functions": 5, @@ -68,7 +68,7 @@ "JSONHTMLTests": 81, "JSONBridgesTests": 10, "JSONCLITests": 88, - "JSONNavigationTests": 36 + "JSONNavigationTests": 37 }, "wasi_root_functions": { "JSONTests": 190, @@ -79,7 +79,7 @@ "JSONIntegrationTests": 32, "JSONHTMLTests": 81, "JSONBridgesTests": 10, - "JSONNavigationTests": 36 + "JSONNavigationTests": 37 }, "wasi_parameterized_invocations": 288, "portable_shared_state_functions": { diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift index 9cb352a..3407260 100644 --- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift +++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift @@ -2,6 +2,6 @@ // Generated by Scripts/generate_cli_identity.py; excludes this file. // SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit. package enum CLIBuildIdentity { - package static let revision = "sha256:21ad62ecd2a10b747b0b45105f85251213e8d49dae0a9c05fabbbfb4c46868ff" + package static let revision = "sha256:83bc9a9a3fa1b647d3dd655ad60ab4798629450913c2b43fd45b3bf509d72d4b" package static let kind = "source-sha256" } diff --git a/Sources/JSONNavigation/JSONTraversal.swift b/Sources/JSONNavigation/JSONTraversal.swift index 9ef4a8e..97228ed 100644 --- a/Sources/JSONNavigation/JSONTraversal.swift +++ b/Sources/JSONNavigation/JSONTraversal.swift @@ -88,6 +88,11 @@ public struct JSONTraversal: Sendable { try state.meter.require(.pointerTokens, UInt64(location.count)) var path = JSONReadPath(scope: scope, components: [], bytes: 0) for part in location { + // Caller-supplied query keys have the same admission as a typed + // read-cursor key; generated child keys were admitted with the root. + if case .key(let key) = part { + try JSONAdmission.text(key.string, key: true, meter: &state.meter) + } path = try scope.append(part, to: path.components, state: state) _ = try scope.check() } diff --git a/Tests/JSONNavigationTests/NavigationBoundaryTests.swift b/Tests/JSONNavigationTests/NavigationBoundaryTests.swift index 8c3b2ac..8668968 100644 --- a/Tests/JSONNavigationTests/NavigationBoundaryTests.swift +++ b/Tests/JSONNavigationTests/NavigationBoundaryTests.swift @@ -238,3 +238,17 @@ private func navigationFailure(_ body: () throws -> Void) -> JSONFailure? { #expect(navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect() }?.category == .resource) #expect(try JSONTraversal(document: document).collect().result.summary.completion == .complete) } + +@Test func navigationRequestedKeyAdmissionMatchesCoreCursor() throws { + let document = JSONDocument(root: .object(try JSONObject())) + for metric: JSONResourceMetric in [.keyBytes, .aggregateTextBytes] { + var limits = JSONLimits.balanced; limits[metric] = 1 + let key = JSONKey("long") + let core = navigationFailure { _ = try document.read(limits: limits) { try $0[key].requiredValue() } } + #expect(core?.category == .resource && core?.resource?.metric == metric) + for selection: JSONTraversalScope in [.subtree(at: [.key(key)]), .virtual(roots: [[.key(key)]])] { + let failure = navigationFailure { _ = try JSONTraversal(document: document, limits: limits).collect(scope: selection) } + #expect(failure == core) + } + } +} From d4730f14d0d68105f7edd9e9273d0fb0430d443c Mon Sep 17 00:00:00 2001 From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com> Date: Thu, 15 Oct 2026 23:56:48 +0200 Subject: [PATCH 3/6] Preserve exact DocC catalogue names in verified transport Replay 4403b9a from PR #34 without altering source semantics. --- .../StructuralIntelligence/Progress.json | 29 ++++++++++++- Scripts/qualify_navigation.py | 42 ++++++++++++++++++- 2 files changed, 68 insertions(+), 3 deletions(-) diff --git a/Documentation/Architecture/StructuralIntelligence/Progress.json b/Documentation/Architecture/StructuralIntelligence/Progress.json index 5984415..d2c8313 100644 --- a/Documentation/Architecture/StructuralIntelligence/Progress.json +++ b/Documentation/Architecture/StructuralIntelligence/Progress.json @@ -4,9 +4,34 @@ "implementation_base": "037e735dc6224789e4d04d2c18d15b8d27440198", "packet_sha256": "d758f75d84271a97da515bf7b7ef70de7a61e91fdf949a511fe0fca2708156f8", "N0": "source authorities reconciled; core-only seven-witness consumer and package-scoped one-ledger composition compiled/executed", - "N1": "experimental physical/virtual DFS, bounded collection and 37 focused behavior functions implemented; complete platform qualification in progress", + "N1": "experimental physical/virtual DFS, bounded collection and 37 behavior functions implemented; functional native/Linux/WASI proofs passed at 6e5e5fa; public product promotion remains gated", "public_product": false, "public_products_preserved": 7, "core_semantic_changes": false, - "deferred": ["BFS", "N2", "N3", "P1", "P2", "R1", "R2", "actors", "persistence", "query grammar", "rendering", "CLI verbs", "EngineBench"] + "deferred": [ + "BFS", + "N2", + "N3", + "P1", + "P2", + "R1", + "R2", + "actors", + "persistence", + "query grammar", + "rendering", + "CLI verbs", + "EngineBench" + ], + "qualification": { + "evidence_source": "6e5e5fa7d8c4c7e8ce1c4fe14bb3323f1ddc7ac8", + "native_run": 38781500802, + "native_boundary": "native/safety/docs/archive qualification commands passed; native/archive artifact upload failed on raw DocC colon filenames; preserved as a failed workflow", + "linux_run": 38781500823, + "linux_boundary": "both architectures qualification and safety passed", + "wasi_boundary": "77-command full Node/limited WasmKit lane passed with unchanged source; supplementary 37-function Navigation Debug/Release runs passed in WasmKit", + "mobile_boundary": "iPhoneOS/iPhoneSimulator 27.2 target compilation passed, no device/runtime claim", + "transport_successor": "native catalogue is packaged byte-for-byte into a verified tar/manifest before upload; fresh hosted outcomes and materialized evidence are supplied by PR 34 and the return packet", + "product_review": "experimental target only; no public product or merge/release action" + } } diff --git a/Scripts/qualify_navigation.py b/Scripts/qualify_navigation.py index 57bced7..f1245ff 100644 --- a/Scripts/qualify_navigation.py +++ b/Scripts/qualify_navigation.py @@ -2,13 +2,16 @@ # SPDX-License-Identifier: Apache-2.0 """Compile/run the unpublished N1 module as an external consumer, including negative compiler probes.""" import argparse +import gzip import hashlib +import io import json import pathlib import re import shutil import subprocess import sys +import tarfile ROOT = pathlib.Path(__file__).resolve().parents[1] CHECKS = {'root-empty-key', 'typed-key-index', 'presence', 'scalar-keys', 'native-exact', @@ -18,6 +21,42 @@ 'replacement-invalidates-spans', 'intentional-stop', 'cumulative-quota'} +def pack_catalog(catalog, output): + """Preserve native DocC names inside a verified portable transport envelope.""" + if catalog.is_symlink() or catalog.parent.resolve() != output.resolve(): + raise ValueError('Unexpected generated catalogue location') + records = [] + for path in sorted(catalog.rglob('*')): + if path.is_symlink(): + raise ValueError('Unexpected generated catalogue symlink') + if path.is_file(): + records.append((path.relative_to(output).as_posix(), path.read_bytes())) + if not records: + raise ValueError('Empty experimental DocC catalogue') + archive = output / 'JSONNavigation-DocC.tar.gz' + manifest = output / 'JSONNavigation-DocC.MANIFEST.sha256' + manifest.write_text(''.join(hashlib.sha256(data).hexdigest() + ' ' + name + '\n' for name, data in records)) + with archive.open('wb') as stream: + with gzip.GzipFile(fileobj=stream, mode='wb', filename='', mtime=0) as zipped: + with tarfile.open(fileobj=zipped, mode='w', format=tarfile.GNU_FORMAT) as tar: + for name, data in records: + entry = tarfile.TarInfo(name); entry.size = len(data); entry.mode = 0o644 + tar.addfile(entry, io.BytesIO(data)) + with tarfile.open(archive) as tar: + members = tar.getmembers() + if len(members) != len(records) or len({m.name for m in members}) != len(records): + raise ValueError('DocC transport inventory differs') + for member, (name, data) in zip(members, records): + if not member.isfile() or member.name != name or tar.extractfile(member).read() != data: + raise ValueError('DocC transport bytes differ') + # Only this helper's verified generated tree is removed. Native ':' filenames + # remain intact inside the tar; GitHub artifact transport never sees raw paths. + shutil.rmtree(catalog) + return {'archive': archive.relative_to(ROOT).as_posix(), 'sha256': hashlib.sha256(archive.read_bytes()).hexdigest(), + 'bytes': archive.stat().st_size, 'files': len(records), 'manifest': manifest.relative_to(ROOT).as_posix(), + 'manifest_sha256': hashlib.sha256(manifest.read_bytes()).hexdigest(), 'all_bytes_verified': True} + + def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument('--output', required=True) @@ -104,8 +143,9 @@ def run(name, command, expected=None): run('docc', ['xcrun', 'docc', 'convert', str(ROOT / 'Sources/JSONNavigation/Documentation.docc'), '--additional-symbol-graph-dir', str(graphs), '--output-dir', str(output / 'JSONNavigation.doccarchive'), '--fallback-bundle-identifier', 'org.swift-json.experimental.JSONNavigation', '--warnings-as-errors']) + receipt['compiled_docc'] = pack_catalog(output / 'JSONNavigation.doccarchive', output) receipt['status'] = 'passed' - except (OSError, ValueError, subprocess.TimeoutExpired) as error: + except (OSError, ValueError, tarfile.TarError, subprocess.TimeoutExpired) as error: receipt['status'] = 'failed'; receipt['error'] = str(error) (output / 'receipt.json').write_text(json.dumps(receipt, indent=2) + '\n') print('NAVIGATION_COMPILER ' + receipt['status'] + ' configurations=2 consumer_cases=18 negative_probes=8') From 8199f3c9e44f9a63aee08c5e566f2535a7204ddb Mon Sep 17 00:00:00 2001 From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com> Date: Thu, 15 Oct 2026 23:56:55 +0200 Subject: [PATCH 4/6] Promote opt-in Navigation product with qualified public consumer Replay only 4403b9a..7ee2712 from stacked PR #37, preserving composition and root test inventory. --- AGENTS.md | 6 +- Documentation/Architecture/API.md | 7 +- .../StructuralIntelligence/N1Promotion.md | 25 + .../StructuralIntelligence/N1PublicAudit.json | 500 ++++++++++++++++++ .../SourceReconciliation.md | 22 + Documentation/CI/README.md | 10 + Documentation/DocumentationTooling.md | 10 +- .../Qualification/TestDiscovery.json | 6 +- Documentation/README.md | 3 +- Package.swift | 3 +- README.md | 21 +- Scripts/README.md | 6 +- Scripts/build_documentation.py | 2 +- Scripts/build_documentation_site.py | 8 +- Scripts/ci.py | 4 +- Scripts/documentation_catalog.py | 2 +- Scripts/qualify_navigation.py | 12 +- Scripts/wasm_support.py | 2 +- Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +- .../Documentation.docc/JSONNavigation.md | 6 +- .../Documentation.docc/Traversal.md | 6 + .../JSONNavigationTests/PromotionTests.swift | 36 ++ Tools/NavigationQualification/Package.swift | 9 + Tools/NavigationQualification/README.md | 8 + 24 files changed, 675 insertions(+), 41 deletions(-) create mode 100644 Documentation/Architecture/StructuralIntelligence/N1Promotion.md create mode 100644 Documentation/Architecture/StructuralIntelligence/N1PublicAudit.json create mode 100644 Documentation/Architecture/StructuralIntelligence/SourceReconciliation.md create mode 100644 Tests/JSONNavigationTests/PromotionTests.swift create mode 100644 Tools/NavigationQualification/Package.swift create mode 100644 Tools/NavigationQualification/README.md diff --git a/AGENTS.md b/AGENTS.md index e5e6be1..7e36c49 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,11 +4,11 @@ Swift-JSON requires Swift tools/compiler 6.4+, Swift language mode 6 and macOS/SDK 27+. Read README.md, CONTRIBUTING.md and the relevant contracts in Documentation/Architecture before changing behavior. -- Keep seven public products: JSON, JSONSchema, JSONStreaming, JSONPatch, JSONHTML, - JSONBridges and the swift-json executable. JSONFilterRuntime and JSONCLIEngine are package-private. +- Keep eight public products: JSON, JSONSchema, JSONStreaming, JSONPatch, JSONHTML, + JSONBridges, JSONNavigation and the swift-json executable. JSONFilterRuntime and JSONCLIEngine are package-private. JSONHTML and opt-in JSONBridges depend on JSON only; core never re-exports the bridges. The approved CLI and E2 additions preserve historical product contracts. - JSONNavigation is experimental without a public product until its N1 release gate. + JSONNavigation is an opt-in product candidate depending on JSON only; merge/release remain human-gated. JSONRegexRuntime is package-private. Root package dependencies remain empty. - Preserve exact bounded decimals, decoded Unicode scalar identity, ordered objects with order-independent equality, one strict UTF-8 grammar, native diff --git a/Documentation/Architecture/API.md b/Documentation/Architecture/API.md index 0acd3c2..55b5292 100644 --- a/Documentation/Architecture/API.md +++ b/Documentation/Architecture/API.md @@ -1,7 +1,7 @@ # Public API contract This document describes the implemented public interface. Native compiler symbol -graphs supply the complete declarations in the six DocC catalogs. Examples +graphs supply the complete declarations in the seven DocC catalogs. Examples in the [README](../../README.md) are compiled and executed by the public consumer. ## Products and ownership @@ -9,6 +9,7 @@ in the [README](../../README.md) are compiled and executed by the public consume | Product | Main entry points | Owned implementation boundary | | --- | --- | --- | | JSON | JSON, JSONNumber, JSONObject, JSONParser, JSONWriter, JSONValueEncoder, JSONValueDecoder, JSONPointer, JSONDocument | Shared strict codec, exact decimal model, bounded traversal and native coding | +| JSONNavigation | JSONTraversal, JSONTraversalScope, JSONTraversalOptions | Exact many-node DFS, strict physical/virtual scopes, bounded metadata and explicit coverage; one core read-scope meter | | JSONBridges | BinaryInteger.jsonNumber, FixedWidthInteger/Double/Decimal.init(jsonNumber:), Double.jsonNumberFromRoundTripText | Optional one-delegate numeric conversions; explicit fidelity, no new authority | | JSONSchema | JSONSchemaDocument, JSONSchemaRegistry, JSONSchemaCompiler, CompiledJSONSchema, JSONSchemaValidator | Offline immutable plan, invocation-owned evaluation and private native regex | | JSONStreaming | JSONEventSequence, JSONValueSequence, NDJSONValueSequence, JSONTopLevelArraySequence, JSONAsyncWriter | Exclusive pull state, shared codec, strict framing and awaited handoffs | @@ -17,7 +18,7 @@ in the [README](../../README.md) are compiled and executed by the public consume | JSONHTML | JSONHTMLRenderer, JSONHTMLRenderingOptions, JSONHTMLLimits, JSONHTMLRenderReport, JSONHTMLAnnotation, JSONExportDocument, JSONExportOptions, JSONExportReport | Structured forms/cards and bounded native previews, exact HTML, native Apple transfer, immutable prepared JSON/text/Markdown/static HTML/opt-in styled RTF and cumulative budgets | JSON imports no siblings. JSONSchema depends on JSON and package-private -JSONRegexRuntime. Streaming, Patch, HTML and Bridges depend on JSON. There are seven public products: six libraries and the swift-json executable, +JSONRegexRuntime. Streaming, Patch, HTML and Bridges depend on JSON. There are eight candidate public products: seven libraries and the swift-json executable, and no external root-package dependencies. No HTTP, MCP or documentation-tool dependency is introduced. JSONHTML conditionally uses Apple SDK frameworks for explicit native transfer/PDF preparation; the portable value, schema, Patch and streaming contracts do not acquire UI semantics. @@ -244,3 +245,5 @@ includes `revisionKind: "source-sha256"` and `revision: "sha256:"`. This is SHA-256 of the ordered Package.swift plus Sources content manifest, excluding only the generated identity file itself. It is not a Git commit, binary digest or claim of publication. Generation/checking needs no Git, time, environment or network. + +JSONNavigation product promotion preserves all 56 N1 public declarations and resource semantics. Full-source admission precedes invalid depth/record-option validation, including oversized sources; N1Promotion.md records this preserved precedence. JSON imports no Navigation symbols implicitly. diff --git a/Documentation/Architecture/StructuralIntelligence/N1Promotion.md b/Documentation/Architecture/StructuralIntelligence/N1Promotion.md new file mode 100644 index 0000000..850057c --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/N1Promotion.md @@ -0,0 +1,25 @@ +# N1 public-product candidate decision + +Approved for implementation: preserve all 56 existing public declarations, now +resolvable as the opt-in JSONNavigation library product. N1PublicAudit.json lists +the source-aware per-declaration decisions. New visibility is additive; this is +not a package version, merge approval or a published release. + +The immutable traversal captures its source; paths describe that source, not a +replacement. Mutable options configure each independent invocation. Selection +indices and occurrence ordinals retain their zero-based, first-examined semantics. +Filters affect emission; coverage is per occurrence rather than a union proof. +Errors/cancellation throw; callback computation and retained copies are caller work. +Package read scopes, state, meters and qualification observers remain hidden. + +P07 decision: keep the existing source-admission-before-depth/record-validation +precedence. An oversized source can therefore report its admission resource error +before an invalid option. Small-source invalid options report configuration errors. +No error precedence is changed during promotion. N2 sampling configuration has a +separate pre-admission check because it is a new contract. + +Public SwiftPM consumer Debug/Release, isolation negatives, docs, all platform +lanes and Git-free reproduction must qualify the final source before recommending +release readiness. Actual outcomes accompany the return; this document is an API +review decision, not a claim that unfinished qualification has passed. Draft #34 +and the immutable N1 anchor remain untouched. No PR-ready/merge/release transition. diff --git a/Documentation/Architecture/StructuralIntelligence/N1PublicAudit.json b/Documentation/Architecture/StructuralIntelligence/N1PublicAudit.json new file mode 100644 index 0000000..8ed9871 --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/N1PublicAudit.json @@ -0,0 +1,500 @@ +{ + "parent": "4403b9ab3b0a1fadc57c12a283cfabb255f6237a", + "authored_declarations": 56, + "source_graph_audit": "fresh Swift6.4 baseline compilation; package/private symbols excluded", + "decisions": [ + { + "path": [ + "JSONScopeOverlapPolicy" + ], + "declaration": "enum JSONScopeOverlapPolicy", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONScopeOverlapPolicy", + "orderedOccurrences" + ], + "declaration": "case orderedOccurrences", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONScopeOverlapPolicy", + "rejectAmbiguous" + ], + "declaration": "case rejectAmbiguous", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONScopeOverlapPolicy", + "uniquePhysicalNodes" + ], + "declaration": "case uniquePhysicalNodes", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversal" + ], + "declaration": "struct JSONTraversal", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversal", + "collect(scope:options:maximumRecords:)" + ], + "declaration": "func collect(scope: JSONTraversalScope = .subtree(at: []), options: JSONTraversalOptions = .init(), maximumRecords: Int = 100_000) throws -> JSONOperationReport", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversal", + "document" + ], + "declaration": "let document: JSONDocument", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversal", + "init(document:limits:)" + ], + "declaration": "init(document: JSONDocument, limits: JSONLimits = .balanced)", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversal", + "limits" + ], + "declaration": "let limits: JSONLimits", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversal", + "visit(scope:options:_:)" + ], + "declaration": "func visit(scope: JSONTraversalScope = .subtree(at: []), options: JSONTraversalOptions = .init(), _ visitor: (JSONTraversalHit, JSON) throws -> JSONTraversalAction) throws -> JSONOperationReport", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalAction" + ], + "declaration": "enum JSONTraversalAction", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalAction", + "continue" + ], + "declaration": "case `continue`", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalAction", + "skipDescendants" + ], + "declaration": "case skipDescendants", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalAction", + "stop" + ], + "declaration": "case stop", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCollection" + ], + "declaration": "struct JSONTraversalCollection", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCollection", + "nodes" + ], + "declaration": "let nodes: [JSONTraversalHit]", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCollection", + "summary" + ], + "declaration": "let summary: JSONTraversalSummary", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCompletion" + ], + "declaration": "enum JSONTraversalCompletion", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCompletion", + "complete" + ], + "declaration": "case complete", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCompletion", + "restricted" + ], + "declaration": "case restricted", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalCompletion", + "visitorStopped" + ], + "declaration": "case visitorStopped", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalFailure" + ], + "declaration": "enum JSONTraversalFailure", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalFailure", + "ambiguousRoots(first:second:)" + ], + "declaration": "case ambiguousRoots(first: Int, second: Int)", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalFailure", + "collectionLimitExceeded(limit:)" + ], + "declaration": "case collectionLimitExceeded(limit: Int)", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit" + ], + "declaration": "struct JSONTraversalHit", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit", + "absoluteLocation" + ], + "declaration": "let absoluteLocation: [JSONLocationComponent]", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit", + "kind" + ], + "declaration": "let kind: JSONTraversalKind", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit", + "occurrenceIndex" + ], + "declaration": "let occurrenceIndex: UInt64", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit", + "relativeDepth" + ], + "declaration": "let relativeDepth: Int", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit", + "scopeRootIndex" + ], + "declaration": "let scopeRootIndex: Int", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalHit", + "sourceSpan" + ], + "declaration": "let sourceSpan: JSONSourceSpan?", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind" + ], + "declaration": "enum JSONTraversalKind", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind", + "array" + ], + "declaration": "case array", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind", + "boolean" + ], + "declaration": "case boolean", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind", + "null" + ], + "declaration": "case null", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind", + "number" + ], + "declaration": "case number", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind", + "object" + ], + "declaration": "case object", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalKind", + "string" + ], + "declaration": "case string", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalOptions" + ], + "declaration": "struct JSONTraversalOptions", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalOptions", + "depthLimit" + ], + "declaration": "var depthLimit: Int?", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalOptions", + "emitKinds" + ], + "declaration": "var emitKinds: Set?", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalOptions", + "includeScopeRoot" + ], + "declaration": "var includeScopeRoot: Bool", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalOptions", + "includeSourceSpans" + ], + "declaration": "var includeSourceSpans: Bool", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalOptions", + "init(depthLimit:includeScopeRoot:emitKinds:includeSourceSpans:)" + ], + "declaration": "init(depthLimit: Int? = nil, includeScopeRoot: Bool = true, emitKinds: Set? = nil, includeSourceSpans: Bool = false)", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalScope" + ], + "declaration": "enum JSONTraversalScope", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalScope", + "subtree(at:)" + ], + "declaration": "case subtree(at: [JSONLocationComponent])", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalScope", + "virtual(roots:overlap:)" + ], + "declaration": "case virtual(roots: [[JSONLocationComponent]], overlap: JSONScopeOverlapPolicy = .rejectAmbiguous)", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary" + ], + "declaration": "struct JSONTraversalSummary", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "completion" + ], + "declaration": "var completion: JSONTraversalCompletion { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "deduplicatedCount" + ], + "declaration": "var deduplicatedCount: UInt64 { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "depthLimitedSubtrees" + ], + "declaration": "var depthLimitedSubtrees: UInt64 { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "emittedCount" + ], + "declaration": "var emittedCount: UInt64 { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "maximumDepthVisited" + ], + "declaration": "var maximumDepthVisited: Int { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "prunedSubtrees" + ], + "declaration": "var prunedSubtrees: UInt64 { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "scopeRootCount" + ], + "declaration": "var scopeRootCount: Int { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + }, + { + "path": [ + "JSONTraversalSummary", + "visitedCount" + ], + "declaration": "var visitedCount: UInt64 { get }", + "decision": "keep", + "reason": "Exact source-bound metadata/coverage or existing single-ledger operation; no competing value/path/cursor authority" + } + ] +} diff --git a/Documentation/Architecture/StructuralIntelligence/SourceReconciliation.md b/Documentation/Architecture/StructuralIntelligence/SourceReconciliation.md new file mode 100644 index 0000000..894365e --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/SourceReconciliation.md @@ -0,0 +1,22 @@ +# N1P/N2A source reconciliation + +Research/N1 parent is4403b9ab3b0a1fadc57c12a283cfabb255f6237a, draft PR34. +Fetched main remains037e735dc6224789e4d04d2c18d15b8d27440198. +Successor codex/structural-navigation-n1p-n2a branches from that exact N1 head. +PR34, its history and its worktree are unchanged. PR31 remains64264218dc0073a5738ab557bfa25b3bb9f6fa7a; +PR35 remains73efb65189f3f7e08e878a3b0fb6030f34524e4b at reconciliation. +Both are drafts. Their manifest/CI/docs edits overlap and require a future explicit +integration; none is merged here. Composition's separate owner is not a borrowed +Navigation scope. Core JSONReadScope/CodingState/WorkMeter, exact comparison, +typed selection/path construction and source maps remain unchanged authorities. + +The maintainer-provided NAV-RETURN-1 and independently recorded PR/workflow states +are historical evidence. This wave does not reconstruct or independently certify +the unavailable250MB control-tower bundle. Fresh commands and source-bound receipts +qualify this successor. Private evidence remains under ignored Artifacts. + +Approved: N1 additive product candidate, unchanged56 public declarations and P07 +precedence; package-only borrowing proof; N2 only after its gate. Rejected: opening +composition/read/Pointer meters inside owned callbacks, unmetered String maps and +partial success on errors. Deferred: snapshot/projection/persistence/BFS/CLI/HTML, +DSL/OpenAPI integration, EngineBench and release/merge actions. diff --git a/Documentation/CI/README.md b/Documentation/CI/README.md index 2b5c044..f584955 100644 --- a/Documentation/CI/README.md +++ b/Documentation/CI/README.md @@ -1,5 +1,15 @@ # Native CI +## N1 product-candidate increment + +Promotion adds the JSONNavigation library product after a 56-declaration audit, +with a real external SwiftPM consumer in Debug/Release. Two new P07 tests preserve +source-admission-first precedence. Focused discovery is now 39 functions; all37 +N1 identities remain. Exact roots are 585 native/569 Linux and 481 WASI, with the +same325/288 parameter inventories and42/41 shared functions. Seven public DocC +catalogues include Navigation; private borrowed interfaces are never public API. +Actual receipts establish passes. Earlier stage counts below retain their source. + ## Experimental N1 branch The Structural Intelligence branch adds exactly 37 Navigation functions and one diff --git a/Documentation/DocumentationTooling.md b/Documentation/DocumentationTooling.md index b46024c..c3194fb 100644 --- a/Documentation/DocumentationTooling.md +++ b/Documentation/DocumentationTooling.md @@ -1,7 +1,7 @@ # Building and maintaining the DocC catalogs Swift source comments are the API documentation authority. Native compiler symbol -graphs supply the declarations and links for six Swift DocC catalogs. Each catalog +graphs supply the declarations and links for seven Swift DocC catalogs. Each catalog combines the product reference with curated Topics and a focused usage guide. See the [Swift DocC documentation](https://www.swift.org/documentation/docc/). @@ -19,7 +19,7 @@ python3 Scripts/ci.py --lane docs The lane records native preflight, extracts fresh public symbol graphs, requires summaries and named-parameter documentation on every authored public declaration, checks README Swift fences against their executable consumer source, runs that -consumer, and compiles all six DocC catalogs with warnings as errors. Zero public +consumer, and compiles all seven DocC catalogs with warnings as errors. Zero public symbols, missing comments, stale examples or a failed catalog fail the lane. | Output | Purpose | @@ -33,9 +33,9 @@ symbols, missing comments, stale examples or a failed catalog fail the lane. | `.build/handoff/docc/symbols/` | Public compiler symbol graphs grouped by product | | `.build/handoff/docc/coverage.json` | Counts and documentation-coverage failures | | `.build/handoff/docc/receipt.json` | Commands, toolchain, counts, exits and diagnostics | -| `.build/handoff/docc/compiled-catalogs.tar.gz` | All six compiled catalogs with their exact native filenames | +| `.build/handoff/docc/compiled-catalogs.tar.gz` | All seven compiled catalogs with their exact native filenames | | `.build/handoff/docc/compiled-catalogs.MANIFEST.sha256` | Independently verifiable hashes for every archived catalog file | -| `.build/handoff/pages/site/` | Static site, landing page and all six references under the project base path | +| `.build/handoff/pages/site/` | Static site, landing page and all seven references under the project base path | | `.build/handoff/pages/receipt.json` | Native transformations, input/output hashes and route/asset verification | Open a `.doccarchive` in Xcode for the compiled reference. To preview a source @@ -64,7 +64,7 @@ The manifest and receipt preserve file-level and archive-level digests. The docs lane uses native `docc process-archive transform-for-static-hosting` to prepare each qualified archive under `/Swift-JSON/`. A shared landing -page links all six references and guides. The build verifies product/guide routes, +page links all seven references and guides. The build verifies product/guide routes, local HTML links and assets, rejects private-path/credential markers and symlinks, and records a file manifest. Only the generated site is packaged for Pages; source graphs, command logs and private maintainer records are not website content. diff --git a/Documentation/Qualification/TestDiscovery.json b/Documentation/Qualification/TestDiscovery.json index 1071562..a173776 100644 --- a/Documentation/Qualification/TestDiscovery.json +++ b/Documentation/Qualification/TestDiscovery.json @@ -10,7 +10,7 @@ "JSONHTMLTests": 97, "JSONBridgesTests": 10, "JSONCLITests": 88, - "JSONNavigationTests": 37 + "JSONNavigationTests": 39 }, "parameterized_invocations": 325, "composition_external_functions": 5, @@ -68,7 +68,7 @@ "JSONHTMLTests": 81, "JSONBridgesTests": 10, "JSONCLITests": 88, - "JSONNavigationTests": 37 + "JSONNavigationTests": 39 }, "wasi_root_functions": { "JSONTests": 190, @@ -79,7 +79,7 @@ "JSONIntegrationTests": 32, "JSONHTMLTests": 81, "JSONBridgesTests": 10, - "JSONNavigationTests": 37 + "JSONNavigationTests": 39 }, "wasi_parameterized_invocations": 288, "portable_shared_state_functions": { diff --git a/Documentation/README.md b/Documentation/README.md index 2c7ee0d..84411d4 100644 --- a/Documentation/README.md +++ b/Documentation/README.md @@ -6,7 +6,7 @@ behavioral contracts and native qualification procedure. ## API references and guided reading -The six native DocC catalogs combine inline API documentation with focused guides: +The seven candidate native DocC catalogs combine inline API documentation with focused guides: | Product | Catalog source | Guide | | --- | --- | --- | @@ -15,6 +15,7 @@ The six native DocC catalogs combine inline API documentation with focused guide | JSONStreaming | [JSONStreaming](../Sources/JSONStreaming/Documentation.docc/JSONStreaming.md) | [Completion and backpressure](../Sources/JSONStreaming/Documentation.docc/CompletionAndBackpressure.md) | | JSONHTML | [JSONHTML](../Sources/JSONHTML/Documentation.docc/JSONHTML.md) | [Formatted rendering](../Sources/JSONHTML/Documentation.docc/FormattedRendering.md) | | JSONBridges | [JSONBridges](../Sources/JSONBridges/Documentation.docc/JSONBridges.md) | [Numeric fidelity](../Sources/JSONBridges/Documentation.docc/NumericFidelity.md) | +| JSONNavigation | [JSONNavigation](../Sources/JSONNavigation/Documentation.docc/JSONNavigation.md) | [Exact traversal](../Sources/JSONNavigation/Documentation.docc/Traversal.md) | | JSONPatch | [JSONPatch](../Sources/JSONPatch/Documentation.docc/JSONPatch.md) | [Choosing and applying patches](../Sources/JSONPatch/Documentation.docc/ApplyingPatches.md) | [Browse the public documentation](https://thorfuchs.github.io/Swift-JSON/) or diff --git a/Package.swift b/Package.swift index 4170c9c..4e78f36 100644 --- a/Package.swift +++ b/Package.swift @@ -12,11 +12,12 @@ let package = Package( .library(name: "JSONPatch", targets: ["JSONPatch"]), .library(name: "JSONHTML", targets: ["JSONHTML"]), .library(name: "JSONBridges", targets: ["JSONBridges"]), + .library(name: "JSONNavigation", targets: ["JSONNavigation"]), .executable(name: "swift-json", targets: ["JSONCLI"]) ], targets: [ .target(name: "JSON"), - // Experimental N1 target. Product promotion requires separate release-gate review. + // Opt-in N1 product candidate; promotion does not change core navigation semantics. .target(name: "JSONNavigation", dependencies: ["JSON"]), .testTarget(name: "JSONNavigationTests", dependencies: ["JSONNavigation", "JSON"], exclude: ["Fixtures"]), .target(name: "JSONBridges", dependencies: ["JSON"]), diff --git a/README.md b/README.md index 4005f31..b14fc75 100644 --- a/README.md +++ b/README.md @@ -17,9 +17,9 @@ Maintained by [Thorsten Fuchs](https://github.com/ThorFuchs). Swift-JSON follows semantic versioning. Use a released version and retain `Package.resolved` for reproducible builds; `main` tracks ongoing development. -This feature branch also contains an [experimental structural traversal target](Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md). -It preserves the seven released products. JSONNavigation product promotion remains -gated; the [compiled traversal guide](Sources/JSONNavigation/Documentation.docc/Traversal.md) +This feature branch also contains an [opt-in structural traversal product candidate](Documentation/Architecture/StructuralIntelligence/N1Promotion.md). +It preserves the seven released products and proposes JSONNavigation as the eighth. +Merge and release remain human-gated; the [compiled traversal guide](Sources/JSONNavigation/Documentation.docc/Traversal.md) describes its many-node scope, exact locations, finite accounting and coverage. Published **1.5.0** provides six libraries and the CLI. This development branch @@ -77,6 +77,7 @@ Choose the products your application needs: | **JSONHTML** | Exact HTML and immutable exports since 1.3.0; structured forms/cards and bounded previews on unreleased main | | **swift-json** | Bounded command-line JSON workflows (available since 1.5.0) | | **JSONBridges** | Opt-in native integer, Double and Foundation Decimal conversions (available in the 1.4.0 tag and 1.5.0 release) | +| **JSONNavigation** | Unreleased candidate: exact many-node DFS, physical/virtual scopes, finite metadata and explicit coverage | | **JSONPatch** | Staged RFC 6902 JSON Patch and RFC 7396 Merge Patch | `JSONSchema`, `JSONStreaming`, `JSONPatch`, `JSONHTML` and opt-in `JSONBridges` build on `JSON`. The package-private @@ -456,7 +457,7 @@ The [documentation index](Documentation/README.md) links the behavior contracts. provide public API references, focused guides, parameter descriptions and failure contracts, compiled from the native Swift public symbol graphs. -Build all six catalogs on a supported native host: +Build all seven candidate catalogs on a supported native host: ```sh python3 Scripts/ci.py --lane docs @@ -564,3 +565,15 @@ cumulative reports. It makes no requests and never rewrites optional properties, Incompatibility exits1; operational errors remain errors. Apple schema/model bridges are separate. See the [provider contract](Documentation/Architecture/Release1.4/Providers.md) and [compiled public consumer](Tools/CLIQualification/Sources/ProviderConsumer/Main.swift). + +## Exact structural navigation (unreleased candidate) + +`JSONNavigation` adds many-node DFS preorder over a captured canonical document, +with insertion-order object members, increasing array indices and exact typed +locations. Physical and virtual scopes, explicit overlap/stop/pruning policies, +bounded metadata collection and coverage share one finite cumulative core budget. +Add the JSONNavigation product and import it explicitly. Existing one-node reads +remain in JSON. Full source admission precedes traversal option validation. +See the [compiled traversal guide](Sources/JSONNavigation/Documentation.docc/Traversal.md) +and [real external SwiftPM consumer](Tools/NavigationQualification). This branch +prepares eight products; published 1.5.0 and its seven-product contract are unchanged. diff --git a/Scripts/README.md b/Scripts/README.md index 26c3e7a..50d07a0 100644 --- a/Scripts/README.md +++ b/Scripts/README.md @@ -9,12 +9,12 @@ separate from the dependency-free library. Native lanes fail on unsupported host | `python3 Scripts/preflight.py` | Actual native OS/compiler/SDK receipt | | `python3 Scripts/ci.py --lane native` | Manifests, builds, Swift Testing, consumers and suites | | `python3 Scripts/ci.py --lane safety` | Address/Thread Sanitizer and strict memory safety | -| `python3 Scripts/ci.py --lane navigation` | Experimental N1 exact discovery, Debug/Release, strict-memory and external compiler/consumer/DocC proof; no product promotion | -| `python3 Scripts/ci.py --lane linux` | Actual Linux preflight, all seven products, exact root/cap/corpus/consumer counts | +| `python3 Scripts/ci.py --lane navigation` | N1 product exact discovery, Debug/Release, strict-memory and real external SwiftPM consumer/compiler/DocC proof; no release action | +| `python3 Scripts/ci.py --lane linux` | Actual Linux preflight, all eight candidate products, exact root/cap/corpus/consumer counts | | `python3 Scripts/ci.py --lane linux-safety` | Linux Address/Thread Sanitizer and strict memory safety | | `python3 Scripts/ci.py --lane wasm` | Matched Swift 6.4 WASI debug/release, exact root/corpus/cap/consumer counts, two-host public parity and structured resource failures | | `python3 Scripts/qualify_linux_reproduction.py --output PATH` | Actual source archive, byte-equal rebuild, clean extraction, build/tests and public consumer | -| `python3 Scripts/ci.py --lane docs` | Six DocC catalogs, authored API/parameter coverage, executed README examples and verified static Pages site | +| `python3 Scripts/ci.py --lane docs` | Seven DocC catalogs, authored API/parameter coverage, executed README examples and verified static Pages site | | `python3 Scripts/build_documentation_site.py` | Transform already-qualified DocC archives and verify static routes/assets | | `python3 Scripts/qualification_watchdog.py --timeout 900 --output PATH -- COMMAND` | Process-group timeout and native receipt | | `python3 Scripts/archive_candidate.py --output PATH` | Public source archive with exact hashes and byte-equal rebuild | diff --git a/Scripts/build_documentation.py b/Scripts/build_documentation.py index 779a5a3..9bdd2ff 100644 --- a/Scripts/build_documentation.py +++ b/Scripts/build_documentation.py @@ -60,7 +60,7 @@ def package_catalogs(output): 'bytes': archive.stat().st_size, 'verified_files': len(records), 'manifest': str(manifest.relative_to(ROOT)), 'manifest_sha256': hashlib.sha256(manifest.read_bytes()).hexdigest(), - 'policy': 'Six native DocC archives; exact internal filenames, sorted entries and normalized tar/gzip metadata.'} + 'policy': 'Seven native DocC archives; exact internal filenames, sorted entries and normalized tar/gzip metadata.'} def main(): diff --git a/Scripts/build_documentation_site.py b/Scripts/build_documentation_site.py index 83f9070..ccc2f03 100644 --- a/Scripts/build_documentation_site.py +++ b/Scripts/build_documentation_site.py @@ -25,14 +25,16 @@ 'JSONStreaming': ('Completion and backpressure', 'completionandbackpressure'), 'JSONPatch': ('Choosing and applying patches', 'applyingpatches'), 'JSONHTML': ('Exact interactive HTML', 'interactiverendering'), - 'JSONBridges': ('Opt-in native conversions', 'numericfidelity')} + 'JSONBridges': ('Opt-in native conversions', 'numericfidelity'), + 'JSONNavigation': ('Exact bounded structural traversal', 'traversal')} DESCRIPTIONS = { 'JSON': 'Exact values, strict UTF-8, native Codable, Pointer, presence and finite resource reports.', 'JSONSchema': 'Draft 2020-12, explicit offline registries, immutable plans, annotations and bounded regex.', 'JSONStreaming': 'Pull events and values, strict NDJSON, root-array completion and awaited output chunks.', 'JSONPatch': 'Transactional RFC 6902 JSON Patch and RFC 7396 Merge Patch with exact value semantics.', 'JSONHTML': 'Exact standalone HTML with offline disclosure/search and native-byte copy/download.', - 'JSONBridges': 'Opt-in trusted integer construction and policy-explicit Double and Foundation Decimal projection.'} + 'JSONBridges': 'Opt-in trusted integer construction and policy-explicit Double and Foundation Decimal projection.', + 'JSONNavigation': 'Exact typed DFS with physical and virtual scopes, explicit coverage and one cumulative budget.'} class Links(HTMLParser): @@ -99,7 +101,7 @@ def landing(commit):
Get startedGitHubSupport
Public Swift documentation

Exact JSON. Clear contracts.

-

Keep decimal values exact, Unicode scalar keys distinct and resource use explicit. Explore six Swift libraries that carry those semantics from input bytes to validated, transformed and encoded output.

+

Keep decimal values exact, Unicode scalar keys distinct and resource use explicit. Explore seven Swift libraries that carry those semantics from input bytes to validated, transformed and encoded output.

Swift 6.4+ · Swift language mode 6 · macOS/SDK 27+ · No package dependencies

{''.join(cards)}

Start with the contract

diff --git a/Scripts/ci.py b/Scripts/ci.py index 309c2fa..d99640a 100644 --- a/Scripts/ci.py +++ b/Scripts/ci.py @@ -210,8 +210,8 @@ def check_result(command, code, log, events=None): passed = data['observed'] == data['passed'] == expected and data['failed'] == data['policy'] == data['blocked'] == 0 if command == ['swift', 'package', 'dump-package'] and passed: d = json.loads(log) - passed = sorted(p['name'] for p in d['products']) == ['JSON', 'JSONBridges', 'JSONHTML', 'JSONPatch', 'JSONSchema', 'JSONStreaming', 'swift-json'] and not d['dependencies'] and d['swiftLanguageVersions'] == ['6'] and d['toolsVersion']['_version'] == '6.4.0' and d['platforms'] == [{'options': [], 'platformName': 'macos', 'version': '27.0'}, {'options': [], 'platformName': 'ios', 'version': '27.0'}] - passed = passed and {p['name']: p['targets'] for p in d['products']} == dict({name: [name] for name in ['JSON', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONBridges']}, **{'swift-json': ['JSONCLI']}) + passed = sorted(p['name'] for p in d['products']) == ['JSON', 'JSONBridges', 'JSONHTML', 'JSONNavigation', 'JSONPatch', 'JSONSchema', 'JSONStreaming', 'swift-json'] and not d['dependencies'] and d['swiftLanguageVersions'] == ['6'] and d['toolsVersion']['_version'] == '6.4.0' and d['platforms'] == [{'options': [], 'platformName': 'macos', 'version': '27.0'}, {'options': [], 'platformName': 'ios', 'version': '27.0'}] + passed = passed and {p['name']: p['targets'] for p in d['products']} == dict({name: [name] for name in ['JSON', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONBridges', 'JSONNavigation']}, **{'swift-json': ['JSONCLI']}) targets = {t['name']: t for t in d['targets']} passed = passed and set(targets) == {'JSON', 'JSONNavigation', 'JSONNavigationTests', 'JSONIntegrationTests', 'JSONBridges', 'JSONBridgesTests', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONRegexRuntime', 'JSONTests', 'JSONSchemaTests', 'JSONStreamingTests', 'JSONPatchTests', 'JSONHTMLTests', 'JSONRegexRuntimeTests', 'JSONFilterRuntime', 'JSONCLIEngine', 'JSONCLI', 'JSONCLITests'} and targets['JSONHTML']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONBridges']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONNavigation']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONNavigationTests']['dependencies'] == [{'byName': ['JSONNavigation', None]}, {'byName': ['JSON', None]}] and targets['JSONFilterRuntime']['dependencies'] == [{'byName': ['JSON', None]}] and targets['JSONCLI']['type'] == 'executable' if command == ['swift', 'package', 'show-dependencies', '--format', 'json'] and passed: diff --git a/Scripts/documentation_catalog.py b/Scripts/documentation_catalog.py index add1f6e..9458193 100644 --- a/Scripts/documentation_catalog.py +++ b/Scripts/documentation_catalog.py @@ -7,7 +7,7 @@ import urllib.parse ROOT = pathlib.Path(__file__).resolve().parents[1] -PRODUCTS = ('JSON', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONBridges') +PRODUCTS = ('JSON', 'JSONSchema', 'JSONStreaming', 'JSONPatch', 'JSONHTML', 'JSONBridges', 'JSONNavigation') def public_symbols(graph_dir): diff --git a/Scripts/qualify_navigation.py b/Scripts/qualify_navigation.py index f1245ff..4d200fe 100644 --- a/Scripts/qualify_navigation.py +++ b/Scripts/qualify_navigation.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: Apache-2.0 -"""Compile/run the unpublished N1 module as an external consumer, including negative compiler probes.""" +"""Qualify the N1 library product with a real external SwiftPM consumer, including negative compiler probes.""" import argparse import gzip import hashlib @@ -64,7 +64,7 @@ def main(): output = ROOT / args.output output.mkdir(parents=True, exist_ok=True) receipt = {'schema_version': 1, 'status': 'running', 'commands': [], - 'scope': 'External Swift module API proof; JSONNavigation is not a released SwiftPM product', + 'scope': 'External SwiftPM product/API proof; candidate source is not a published release', 'consumer_sha256': hashlib.sha256((ROOT / 'Tools/NavigationQualification/Consumer.swift').read_bytes()).hexdigest()} def run(name, command, expected=None): @@ -95,9 +95,7 @@ def run(name, command, expected=None): raise ValueError('Missing built module objects: ' + module) objects.extend(str(p) for p in selected) common = ['swiftc', '-swift-version', '6', '-warnings-as-errors', '-I', str(modules), '-package-name', 'navigation_external_consumer'] - executable = output / ('consumer-' + config) - run(config + '-consumer-compile', common + ['-parse-as-library', str(ROOT / 'Tools/NavigationQualification/Consumer.swift'), *objects, '-o', str(executable)]) - text = run(config + '-consumer-run', [str(executable)]) + text = run(config + '-swiftpm-consumer', ['swift', 'run', '-c', config, '--package-path', 'Tools/NavigationQualification', 'NavigationConsumer']) observed = re.findall(r'^NAVIGATION_CHECK (\S+) passed$', text, re.M) if len(observed) != 18 or set(observed) != CHECKS or 'NAVIGATION_CONSUMER passed cases=18' not in text: raise ValueError('Missing/duplicate public consumer outcomes') @@ -134,7 +132,7 @@ def run(name, command, expected=None): for path in (ROOT / '.build/out/symbolgraph').glob('JSONNavigation*.symbols.json'): shutil.copyfile(path, graphs / path.name) import documentation_catalog - # This local audit does not promote an experimental module to the public site. + # The public site separately qualifies the complete seven-library inventory. documentation_catalog.PRODUCTS = ('JSONNavigation',) coverage = documentation_catalog.verify_comments(documentation_catalog.public_symbols(graphs)) receipt['experimental_api_coverage'] = coverage @@ -142,7 +140,7 @@ def run(name, command, expected=None): raise ValueError('Undocumented experimental API: ' + str(coverage['errors'])) run('docc', ['xcrun', 'docc', 'convert', str(ROOT / 'Sources/JSONNavigation/Documentation.docc'), '--additional-symbol-graph-dir', str(graphs), '--output-dir', str(output / 'JSONNavigation.doccarchive'), - '--fallback-bundle-identifier', 'org.swift-json.experimental.JSONNavigation', '--warnings-as-errors']) + '--fallback-bundle-identifier', 'org.swift-json.JSONNavigation', '--warnings-as-errors']) receipt['compiled_docc'] = pack_catalog(output / 'JSONNavigation.doccarchive', output) receipt['status'] = 'passed' except (OSError, ValueError, tarfile.TarError, subprocess.TimeoutExpired) as error: diff --git a/Scripts/wasm_support.py b/Scripts/wasm_support.py index 3a068d5..102f0d0 100644 --- a/Scripts/wasm_support.py +++ b/Scripts/wasm_support.py @@ -68,7 +68,7 @@ def binary(scratch, config, name): if preamble and not re.fullmatch(r"Another instance of SwiftPM .*waiting until that process has finished execution\.\.\.", preamble): raise ValueError('Unexpected manifest preamble: ' + preamble) manifest = json.loads(manifest_text[start:]) - if manifest['dependencies'] or sorted(p['name'] for p in manifest['products']) != ['JSON', 'JSONBridges', 'JSONHTML', 'JSONPatch', 'JSONSchema', 'JSONStreaming', 'swift-json']: + if manifest['dependencies'] or sorted(p['name'] for p in manifest['products']) != ['JSON', 'JSONBridges', 'JSONHTML', 'JSONNavigation', 'JSONPatch', 'JSONSchema', 'JSONStreaming', 'swift-json']: raise ValueError('Changed public products/dependencies') def invoke(engine, path, mode='command', arguments=(), repo=True, resources=True): if engine == 'node': diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift index 3407260..e917a08 100644 --- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift +++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift @@ -2,6 +2,6 @@ // Generated by Scripts/generate_cli_identity.py; excludes this file. // SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit. package enum CLIBuildIdentity { - package static let revision = "sha256:83bc9a9a3fa1b647d3dd655ad60ab4798629450913c2b43fd45b3bf509d72d4b" + package static let revision = "sha256:f14610adda289c20c22b2b5c6caa56623f09c6ab8a5532115c9dd15084a8ce92" package static let kind = "source-sha256" } diff --git a/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md b/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md index 9e65c2d..f76277c 100644 --- a/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md +++ b/Sources/JSONNavigation/Documentation.docc/JSONNavigation.md @@ -2,9 +2,9 @@ Exact, synchronous, bounded exploration of many nodes in one canonical JSON source. -This is an experimental N1 target, **not a released SwiftPM product**. The package's -seven public products remain unchanged. Product promotion requires the recorded -correctness, resource, compiler, platform and API review gate. +JSONNavigation is an opt-in library product candidate on this development branch. +Add its SwiftPM product and import it explicitly; core JSON does not re-export it. +Product visibility does not establish a published version or approval to merge. The core JSON module continues to own one-node lookup, exact numbers and keys, presence and RFC 6901 Pointer. Use ``JSONTraversal`` for deterministic DFS preorder, diff --git a/Sources/JSONNavigation/Documentation.docc/Traversal.md b/Sources/JSONNavigation/Documentation.docc/Traversal.md index b04f9bc..5f01f24 100644 --- a/Sources/JSONNavigation/Documentation.docc/Traversal.md +++ b/Sources/JSONNavigation/Documentation.docc/Traversal.md @@ -74,3 +74,9 @@ an unqualified hash index. Complete path copies are bounded by the existing read scope; deep frontiers can use quadratic path metadata. No EngineBench speed, linear total-time, zero-copy or fixed-memory claim is made. Classification, snapshots, projections, persistence, async actors, query syntax and rendering are later gates. + +Full captured-source admission precedes validation of depth and collection options. +An oversized source may therefore refuse admission before a negative option is +reported. A valid small source with a negative option throws invalidLimits. +The real external SwiftPM consumer resolves the JSONNavigation product in Debug +and Release; package scopes/meters and the owned context remain inaccessible. diff --git a/Tests/JSONNavigationTests/PromotionTests.swift b/Tests/JSONNavigationTests/PromotionTests.swift new file mode 100644 index 0000000..e6be180 --- /dev/null +++ b/Tests/JSONNavigationTests/PromotionTests.swift @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +import JSON +import JSONNavigation + +@Test func navigationPromotionDepthValidationPreservesAdmissionPrecedence() throws { + for oversized in [false, true] { + var limits = JSONLimits.balanced; limits[.nodes] = 1 + let value: JSON = oversized ? .array([.null]) : .null + do { + _ = try JSONTraversal(document: .init(root: value), limits: limits).visit(options: .init(depthLimit: -1)) { _, _ in + Issue.record("Invalid invocation must not call visitor"); return .continue + } + Issue.record("Expected refusal") + } catch let failure as JSONFailure { + #expect(failure.category == (oversized ? .resource : .configuration)) + #expect(failure.code == (oversized ? .limitExceeded : .invalidLimits)) + #expect(failure.resource?.metric == (oversized ? .nodes : nil)) + } + } +} + +@Test func navigationPromotionRecordValidationPreservesAdmissionPrecedence() throws { + for oversized in [false, true] { + var limits = JSONLimits.balanced; limits[.nodes] = 1 + let value: JSON = oversized ? .array([.null]) : .null + do { + _ = try JSONTraversal(document: .init(root: value), limits: limits).collect(maximumRecords: -1) + Issue.record("Expected refusal") + } catch let failure as JSONFailure { + #expect(failure.category == (oversized ? .resource : .configuration)) + #expect(failure.code == (oversized ? .limitExceeded : .invalidLimits)) + #expect(failure.resource?.metric == (oversized ? .nodes : nil)) + } + } +} diff --git a/Tools/NavigationQualification/Package.swift b/Tools/NavigationQualification/Package.swift new file mode 100644 index 0000000..0437236 --- /dev/null +++ b/Tools/NavigationQualification/Package.swift @@ -0,0 +1,9 @@ +// swift-tools-version: 6.4 +// SPDX-License-Identifier: Apache-2.0 +import PackageDescription +let package = Package(name: "NavigationQualification", platforms: [.macOS("27.0"), .iOS("27.0")], + dependencies: [.package(name: "Swift-JSON", path: "../..")], + targets: [.executableTarget(name: "NavigationConsumer", dependencies: [ + .product(name: "JSON", package: "Swift-JSON"), + .product(name: "JSONNavigation", package: "Swift-JSON") + ], path: ".", exclude: ["README.md"], sources: ["Consumer.swift"])], swiftLanguageModes: [.v6]) diff --git a/Tools/NavigationQualification/README.md b/Tools/NavigationQualification/README.md new file mode 100644 index 0000000..f31ab63 --- /dev/null +++ b/Tools/NavigationQualification/README.md @@ -0,0 +1,8 @@ +# External Navigation consumer + +This separate SwiftPM package resolves JSONNavigation through its real library product, +using the exact source checkout as a path dependency. Run `swift run --package-path +Tools/NavigationQualification NavigationConsumer` and repeat with `-c release`. +The executable checks 18 named exact path/value/coverage/resource outcomes. The +repository compiler lane also checks import/package/cursor isolation and executes +the three public DocC guide examples. No package-scoped meter is imported here. From 134a516cd272d043e622f2f6de624e1ee15dfa82 Mon Sep 17 00:00:00 2001 From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com> Date: Thu, 15 Oct 2026 23:57:02 +0200 Subject: [PATCH 5/6] Retain public promotion history and isolate blocked compiler research Replay 5cf7897 from PR #37. Historical source receipts remain unchanged. --- CHANGELOG.md | 9 +++++++++ Documentation/Architecture/Linux.md | 8 ++++++++ .../StructuralIntelligence/API-DECISIONS.md | 6 ++++++ .../StructuralIntelligence/N1Promotion.md | 7 +++++++ .../PromotionProgress.json | 17 +++++++++++++++++ Documentation/Architecture/WebAssembly.md | 9 +++++++++ Documentation/DocumentationTooling.md | 1 + Sources/JSONCLIEngine/CLIBuildIdentity.swift | 2 +- .../Documentation.docc/Traversal.md | 2 +- 9 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 Documentation/Architecture/StructuralIntelligence/PromotionProgress.json diff --git a/CHANGELOG.md b/CHANGELOG.md index ed17dc6..12b9cd6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -170,3 +170,12 @@ remain unselected; projection and RFC 7464 framing remain deferred. timing sleeps; retain an invocation-local package-only checkpoint observer. - Qualify twenty frozen explicitly rooted downstream schema vectors and public cumulative budgets in a separate consumer; no protocol/runtime dependency added. + +## Unreleased Navigation candidate + +- Propose opt-in JSONNavigation as an eighth SwiftPM product, preserving its56 + exact traversal declarations and cumulative resource semantics. +- Qualify a real external Debug/Release SwiftPM consumer and preserve source-first + error precedence with two explicit regressions. +- N2 inspection remains unimplemented: its separate borrowed-context compiler + proof is blocked. No version/tag/release or automatic merge is implied. diff --git a/Documentation/Architecture/Linux.md b/Documentation/Architecture/Linux.md index dd787ab..7948ead 100644 --- a/Documentation/Architecture/Linux.md +++ b/Documentation/Architecture/Linux.md @@ -1,3 +1,11 @@ +# N1 candidate platform scope + +This additive candidate proposes JSONNavigation as an eighth product (seven +libraries plus CLI), with no external dependencies. Its public SwiftPM consumer +and39 Navigation functions join the unchanged Linux lanes. Both actual ARM64 and +x86_64 full/safety receipts are required for final qualification. The historical +platform text below describes the original seven-product implementation. + # Linux platform contract The six libraries and swift-json executable are qualified for 64-bit diff --git a/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md b/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md index ade93f7..9632899 100644 --- a/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md +++ b/Documentation/Architecture/StructuralIntelligence/API-DECISIONS.md @@ -1,3 +1,9 @@ +# Historical N0/N1 decision record + +This records the experimental4403b9a anchor. The current public-product candidate +is described in [N1Promotion.md](N1Promotion.md); the original decisions below +retain their historical source context. + # N1 exact traversal decisions Scope is N0/N1 only. JSONNavigation depends exclusively on JSON and currently has diff --git a/Documentation/Architecture/StructuralIntelligence/N1Promotion.md b/Documentation/Architecture/StructuralIntelligence/N1Promotion.md index 850057c..1ca40fe 100644 --- a/Documentation/Architecture/StructuralIntelligence/N1Promotion.md +++ b/Documentation/Architecture/StructuralIntelligence/N1Promotion.md @@ -23,3 +23,10 @@ lanes and Git-free reproduction must qualify the final source before recommendin release readiness. Actual outcomes accompany the return; this document is an API review decision, not a claim that unfinished qualification has passed. Draft #34 and the immutable N1 anchor remain untouched. No PR-ready/merge/release transition. + +A0 is independently blocked in the separate e11f93f borrowing spike: Apple Swift6.4 +accepts an escaping borrowed capture at type checking and aborts in SILGenCleanup +during full compilation. A compiler crash is not a valid negative proof. No borrowed +context or JSONInspection code is included in this product candidate. Its executor +is byte-identical to the reviewed N1 anchor. The return preserves exact reproducer +commands, positive/runtime results and failed diagnostic outcomes. diff --git a/Documentation/Architecture/StructuralIntelligence/PromotionProgress.json b/Documentation/Architecture/StructuralIntelligence/PromotionProgress.json new file mode 100644 index 0000000..1faea89 --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/PromotionProgress.json @@ -0,0 +1,17 @@ +{ + "parent": "4403b9ab3b0a1fadc57c12a283cfabb255f6237a", + "initial_product_commit": "7ee2712", + "P1": "56 public declarations preserved; real external SwiftPM Debug/Release18 checks/config,39 regression functions/config passed; full final-source qualification required", + "A0": "separate e11f93f spike: runtime46/config pass; Apple Swift6.4 escape compiler crashes at full code generation; gate BLOCKED", + "A1": "not started per stop condition; JSONInspection target/product absent", + "public_products": 8, + "new_product": "JSONNavigation", + "human_gates": [ + "PR-ready transition", + "merge", + "tag", + "version release", + "publication" + ], + "evidence": "source-bound actual final outcomes accompany the private return; no old250MB bundle reconstructed" +} diff --git a/Documentation/Architecture/WebAssembly.md b/Documentation/Architecture/WebAssembly.md index 5ace53a..c7850af 100644 --- a/Documentation/Architecture/WebAssembly.md +++ b/Documentation/Architecture/WebAssembly.md @@ -1,3 +1,12 @@ +# N1 candidate WASI scope + +The opt-in JSONNavigation product candidate keeps the same full-language Swift6.4 +wasm32-unknown-wasip1 scope and empty dependency graph. Exact discovery is481 +functions/288 parameters, including39 Navigation functions, with all49 core cap +outcomes and existing independent policies. CLI runtime remains a separate scope. +Actual source-bound execution receipts are required; the historical stage counts +below are preserved, not attributed to this candidate. + # WASI command support This additive local branch supports the full-language Swift 6.4 SDK target diff --git a/Documentation/DocumentationTooling.md b/Documentation/DocumentationTooling.md index c3194fb..393047b 100644 --- a/Documentation/DocumentationTooling.md +++ b/Documentation/DocumentationTooling.md @@ -30,6 +30,7 @@ symbols, missing comments, stale examples or a failed catalog fail the lane. | `.build/handoff/docc/JSONHTML.doccarchive` | Exact formatted HTML API and rendering guide | | `.build/handoff/docc/JSONPatch.doccarchive` | Patch API and choosing/applying patches guide | | `.build/handoff/docc/JSONBridges.doccarchive` | Opt-in native numeric conversion and fidelity guide | +| `.build/handoff/docc/JSONNavigation.doccarchive` | Opt-in exact structural traversal and coverage | | `.build/handoff/docc/symbols/` | Public compiler symbol graphs grouped by product | | `.build/handoff/docc/coverage.json` | Counts and documentation-coverage failures | | `.build/handoff/docc/receipt.json` | Commands, toolchain, counts, exits and diagnostics | diff --git a/Sources/JSONCLIEngine/CLIBuildIdentity.swift b/Sources/JSONCLIEngine/CLIBuildIdentity.swift index e917a08..c1d9c13 100644 --- a/Sources/JSONCLIEngine/CLIBuildIdentity.swift +++ b/Sources/JSONCLIEngine/CLIBuildIdentity.swift @@ -2,6 +2,6 @@ // Generated by Scripts/generate_cli_identity.py; excludes this file. // SHA-256 of the ordered Package.swift + Sources content manifest, not a Git commit. package enum CLIBuildIdentity { - package static let revision = "sha256:f14610adda289c20c22b2b5c6caa56623f09c6ab8a5532115c9dd15084a8ce92" + package static let revision = "sha256:bba3e1b374e88d3df8fe4fd5e58423bdfb8982a085754f2d5ad6813830da83db" package static let kind = "source-sha256" } diff --git a/Sources/JSONNavigation/Documentation.docc/Traversal.md b/Sources/JSONNavigation/Documentation.docc/Traversal.md index 5f01f24..e627a8d 100644 --- a/Sources/JSONNavigation/Documentation.docc/Traversal.md +++ b/Sources/JSONNavigation/Documentation.docc/Traversal.md @@ -79,4 +79,4 @@ Full captured-source admission precedes validation of depth and collection optio An oversized source may therefore refuse admission before a negative option is reported. A valid small source with a negative option throws invalidLimits. The real external SwiftPM consumer resolves the JSONNavigation product in Debug -and Release; package scopes/meters and the owned context remain inaccessible. +and Release; package scopes and meters remain inaccessible. From 621f58dae3609f00fa761423550060d78aedd626 Mon Sep 17 00:00:00 2001 From: Thorsten Fuchs <76904008+ThorFuchs@users.noreply.github.com> Date: Thu, 15 Oct 2026 23:58:22 +0200 Subject: [PATCH 6/6] Bind Navigation integration to current composition and CI contracts --- .../Architecture/CompositionQualification.md | 5 +- .../StructuralIntelligence/Integration.md | 55 +++++++++++++++++++ .../StructuralIntelligence/N1Promotion.md | 9 +++ Documentation/CI/README.md | 16 ++++++ 4 files changed, 83 insertions(+), 2 deletions(-) create mode 100644 Documentation/Architecture/StructuralIntelligence/Integration.md diff --git a/Documentation/Architecture/CompositionQualification.md b/Documentation/Architecture/CompositionQualification.md index 20b83de..47f77bc 100644 --- a/Documentation/Architecture/CompositionQualification.md +++ b/Documentation/Architecture/CompositionQualification.md @@ -3,7 +3,8 @@ The synchronous borrowed composition API is described in [Compose native operations](../../Sources/JSON/Documentation.docc/Composition.md). Its public surface and default profiles are unchanged by this qualification work. -The root dependency graph remains empty and the package retains seven products. +The root dependency graph remains empty and the package retains the existing products and adds opt-in JSONNavigation after its +separate public-product gate. The separate [qualification package](../../Tools/CompositionQualification) exercises source-backed parsing, exact Pointer targets, URI identity and explicitly charged @@ -62,7 +63,7 @@ Run python3 Scripts/ci.py --lane composition on the supported native or Linux runner. The lane requires 12 core functions in each build mode, separate public consumers, five external Swift Testing functions in Debug/Release/ASan, all eight scale cells and 11 compiler rejection cases. Exact module/cardinality checks -reject missing tests and misclassified resource failures. The 558 native / 542 +reject missing tests and misclassified resource failures. The reconciled 597 native / 581 Linux root functions, 325 parameter invocations, restored identities and 49 public metric probes retain their existing contracts. diff --git a/Documentation/Architecture/StructuralIntelligence/Integration.md b/Documentation/Architecture/StructuralIntelligence/Integration.md new file mode 100644 index 0000000..8b7d933 --- /dev/null +++ b/Documentation/Architecture/StructuralIntelligence/Integration.md @@ -0,0 +1,55 @@ +# Navigation over composition-enabled main + +The integration base is `30631e8fe60201518c00544c172340ec6576e0d5`, the +protected-main merge of composition PR #35. Original Navigation PR #34 ends at +`4403b9ab3b0a1fadc57c12a283cfabb255f6237a`; its public-product promotion in +stacked PR #37 ends at `5cf78972b279f398894dd4afcbc09dd058889371`. +The integration replays those increments on a new main-based branch. It does +not rewrite either historical head or include unrelated DSL/composition-proof +branches. Original source-bound receipts remain evidence of their own heads. + +## Source and conflict decisions + +| Original increment | Integration commit | Decision | +| --- | --- | --- | +| 94369e1, exact traversal | f83a638 | Add Navigation target, tests and lanes; keep composition lane, checks, extraction notice and test identities | +| 6e5e5fa, scope-key admission | f168e9d | Preserve exact core admission before path staging | +| 4403b9a, DocC transport | d4730f1 | Preserve verified tar transport without renaming native DocC paths | +| 7ee2712, public product | 8199f3c | Add opt-in JSON-only product and external SwiftPM consumer | +| 5cf7897, promotion boundary | 134a516 | Preserve historical decision and isolate failed compiler spike | + +Generated CLI identity is recomputed from final Package.swift and Sources; +neither branch's old stamp is authoritative for the combination. Discovery merges +add Navigation's 39 functions without changing existing module counts. CI retains +composition's public consumers, capacity-characterization policy and normal +negative compiler checks. No test exclusions, quota values or safety flags change. + +Navigation source, its 39 tests, fixture bytes, DocC and external consumer are +byte-identical to 5cf7897. Core JSON, JSONSchema and composition tests/tools are +byte-identical to the fresh base. The 56 authored Navigation declarations retain +their exact shape; source admission/error precedence, DFS/overlap/coverage, +typed paths, cancellation and logical retention remain unchanged. + +## Review and qualification boundary + +The public review accepts explicit opt-in traversal with metadata-only collected +results and synchronous caller work. It adds no cursor, path grammar, mutation, +fresh nested meter or speed claim. All existing products remain available; the +external dependency graph is empty. A0 research and JSONInspection are absent. + +Fresh native, safety, docs, archive, both Linux architectures, WASI and mobile SDK +consumer receipts must bind to this combined source before merge. A green check +on an old stacked head is insufficient. Required main checks and resolved review +threads remain mandatory; only a qualified new integration PR may merge under +the owner's bounded authorization. No release tag or publication follows here. + +PR #31 (DSL) and #36 (composition qualification) retain their separate ownership. +Their CI/discovery/docs overlaps need additive reconciliation when their owners +update their bases. This integration neither rebases nor merges their work. + +## Deferred ownership research + +The original noncopyable borrowed-context escape still requires full codegen +qualification. Its failed `SILGenCleanup` probe is not a valid negative result. +Any recovery and conditional inspector live on a separate experimental branch; +they cannot change this Navigation candidate or borrow historical passes. diff --git a/Documentation/Architecture/StructuralIntelligence/N1Promotion.md b/Documentation/Architecture/StructuralIntelligence/N1Promotion.md index 1ca40fe..1b514c4 100644 --- a/Documentation/Architecture/StructuralIntelligence/N1Promotion.md +++ b/Documentation/Architecture/StructuralIntelligence/N1Promotion.md @@ -1,3 +1,12 @@ +# Reconciled public-product gate + +The historical candidate decision below belongs to 5cf7897. Its no-transition +statement records that earlier authorization boundary. The owner subsequently +authorized one fresh-main Navigation integration and guarded merge after full +current-source qualification and protected review. The reconciliation preserves +all 56 declarations and adds no A0 interface. See [Integration](Integration.md) +for source mapping and the current gate; tagging/publication remain separate. + # N1 public-product candidate decision Approved for implementation: preserve all 56 existing public declarations, now diff --git a/Documentation/CI/README.md b/Documentation/CI/README.md index f584955..01c1973 100644 --- a/Documentation/CI/README.md +++ b/Documentation/CI/README.md @@ -1,5 +1,21 @@ # Native CI +## Reconciled Navigation and composition increment + +Current source retains composition from main and adds the opt-in Navigation +product. The discovery contract requires 597 native functions, 581 Linux functions +and 493 WASI functions, including all 39 Navigation and 12 composition functions. +Parameterized inventories remain 325 native/Linux and 288 WASI. Shared-state +sanitizer expectations are 43 native and 42 Linux. All 126 restored maintenance +identities and all 49 public capacity probes remain mandatory. Full ASan uses the +current root inventory. Native has 33 commands, Linux 29; the separate composition +lane retains its nine commands, five external functions, eight capacity cells and +11 negative compiler cases. Navigation's focused lane retains five commands. +Seven public library DocC catalogues include Navigation. These are acceptance +expectations; source-bound receipts establish actual execution outcomes. + +The following dated sections retain their original source context. + ## N1 product-candidate increment Promotion adds the JSONNavigation library product after a 56-declaration audit,