Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/composition-performance.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Composition performance
on:
pull_request:
paths:
- '.github/workflows/composition-performance.yml'
- 'Scripts/qualify_composition_performance.py'
- 'Scripts/composition_host.py'
- 'Scripts/test_composition_performance.py'
- 'Sources/JSON/Limits/JSONCompositionScope.swift'
- 'Sources/JSON/Limits/JSONWorkMeter.swift'
- 'Tools/CompositionQualification/Sources/CompositionBenchmark/**'
workflow_dispatch:
permissions:
contents: read
concurrency:
group: composition-performance-${{ github.ref }}
cancel-in-progress: false
jobs:
compare:
name: Composition / quiet native Release
runs-on: xcode-27
timeout-minutes: 30
env:
DEVELOPER_DIR: /Applications/Xcode_27.2.app/Contents/Developer
SDKROOT: /Applications/Xcode_27.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
PYTHONDONTWRITEBYTECODE: '1'
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
with:
persist-credentials: false
fetch-depth: 0
- name: Compare exact baseline and candidate on a quiet host
run: python3 -B Scripts/qualify_composition_performance.py --output .build/ci/composition-performance
- name: Preserve actual measurements and eligibility receipts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: composition-performance
if-no-files-found: error
retention-days: 30
path: |
.build/ci/composition-performance/receipt.json
.build/ci/composition-performance/*.log
4 changes: 3 additions & 1 deletion .github/workflows/linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ jobs:
- run: python3 -B Scripts/ci.py --lane '${{ matrix.lane }}'
- name: Public composition and reference imports
if: matrix.lane == 'linux'
run: python3 -B Scripts/ci.py --lane composition
run: |
python3 -B Scripts/ci.py --lane composition
python3 -B Scripts/ci.py --lane composition-proof
- name: Preserve Linux receipts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/native.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
strategy:
fail-fast: false
matrix:
lane: [native, safety, docs, archive]
lane: [native, safety, docs, archive, composition-proof]
env:
DEVELOPER_DIR: /Applications/Xcode_27.2.app/Contents/Developer
PYTHONDONTWRITEBYTECODE: '1'
Expand Down
21 changes: 21 additions & 0 deletions Documentation/Architecture/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,4 +287,25 @@ SHA-256 of the ordered Package.swift plus Sources content manifest, excluding on
the generated identity file itself. It is not a Git commit, binary digest or claim
of publication. Generation/checking needs no Git, time, environment or network.


## Borrowed native composition

`JSON.withComposition(limits:_:)` synchronously borrows a noncopyable,
non-Sendable `JSONCompositionScope` and returns `JSONOperationReport<Result>`
only after healthy finalization. The scope has no public initializer. Public
methods parse a source-backed `Data` document, admit an existing value, construct,
parse, append and resolve native Pointers, charge external work, reserve checked
external logical bytes, checkpoint and tighten limits. Importing JSONSchema adds
`uri(_:relativeTo:)` and `removingFragment(_:)`; JSON has no sibling dependency.
URI operations use the core domain, with no compiled-plan or compilation-work
charge. This slice does not compose schema compilation/evaluation or Codable.

All owned errors latch the original failure. Neither a caught failure, a nested
independent invocation nor later callback failure permits a completed outer
report. Scope handles cannot escape the synchronous borrow or cross suspension.
Native values may be returned; adapter-owned containers must be prepaid. Reports
are ordinary publicly constructible values, not certification of arbitrary
callback work. See [composition qualification](../Qualification/Composition/README.md)
and the JSON DocC composition guide for the exact accounting/ownership contract.

JSONNavigation product promotion preserves all 56 N1 public declarations and resource semantics. Full-source admission precedes invalid depth/record-option validation, including oversized sources; N1Promotion.md records this preserved precedence. JSON imports no Navigation symbols implicitly.
9 changes: 8 additions & 1 deletion Documentation/Architecture/CompositionQualification.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Run python3 Scripts/ci.py --lane composition on the supported native or Linux
runner. The lane requires 12 core functions in each build mode, separate public
consumers, five external Swift Testing functions in Debug/Release/ASan, all eight
scale cells and 11 compiler rejection cases. Exact module/cardinality checks
reject missing tests and misclassified resource failures. The reconciled 597 native / 581
reject missing tests and misclassified resource failures. The reconciled 599 native / 583
Linux root functions, 325 parameter invocations, restored identities and 49 public
metric probes retain their existing contracts.

Expand All @@ -73,6 +73,13 @@ late missing-target/cancellation failures before attempting normal completion.
Finalization must still throw the original native failure. Each report retains
its actual source, environment, command exits, counts and input hashes.

Completion additionally owns a 424-byte counter report and charges 50 work units.
The retained threshold assertions include that independent buffer, and source input
capacity is per document while reported bytes accumulate across all documents.
The original two-argument reservation method is retained; a checked `overhead:`
overload is additive. See [reconciliation](../Qualification/Composition/Reconciliation.md)
for the independent ledger/mutant and compiler proof carried from PR #36.

CI runs the focused lane alongside the existing Native / native and both Linux
architecture jobs. Their names and protection rules remain unchanged. Sanitizer
and full-root lanes continue independently. Linux uses the existing Swift
Expand Down
3 changes: 2 additions & 1 deletion Documentation/Architecture/ExtractionManifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,8 @@
"1.5.1 restores 126 original Apache-2.0 test functions and qualification tools from Swift-JSON 1.2.1 (e8b687f822c577311156b13bf1d15c2d5620b84e), with file hashes and stable function identities in Qualification/Release1.5.1/Restoration.json. The diagnostic omission and Git-free provenance repairs are restored original code; current seven-product harness reconciliation and deterministic CLI identity are original contributions. No new upstream adaptation or dependency.",
"1.6 S3 G7 seeded grammar, scalar-key permutation tests, public replay consumer and strict cardinality/matrix harness are original Apache-2.0 contributions. The 64-schema authored fixture has SHA-256 cb55604183a6db7df5fa968db283e7abeae794277d2b0cef033b0bdb92e615cb. Independent python-jsonschema 4.26.0 is research qualification only, with exact numeric binding; no Python runtime dependency or copied validator/engine is distributed.",
"1.6 S3 G8 materializes the unchanged synthetic corpus per caller access, replacing process-lifetime stored JSON payloads exposed by Linux ASan/LSan. The public process-lifetime consumer and strengthened exact/value-copy test are original Apache-2.0 qualification code; no suppression, runtime fallback, new dependency or corpus/digest change is introduced.",
"Composition reference-load fixtures, serial descriptor staging, public-only adapter, capacity/terminal tests and strict CI receipt checks are original Apache-2.0 code. Generated JSON graphs introduce no upstream protocol fixtures, engine or dependency. Core public APIs, finite profiles and accounting weights are unchanged."
"Composition reference-load fixtures, serial descriptor staging, public-only adapter, capacity/terminal tests and strict CI receipt checks are original Apache-2.0 code. Generated JSON graphs introduce no upstream protocol fixtures, engine or dependency. Core public APIs, finite profiles and accounting weights are unchanged.",
"Composition reconciliation retains PR35 native routes/reference fixtures and adds PR36 original Apache-2.0 report-overlap, per-document capacity/history, independent owned-event/mutant and compiler proofs. No third-party source or root dependency is added."
],
"jsonhtml_export_original_contributions": {
"license": "Apache-2.0",
Expand Down
31 changes: 31 additions & 0 deletions Documentation/Architecture/ResourceLimits.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,34 @@ uninstrumented adapter allocations and process RSS are outside the claim. A publ
JSONOperationReport initializer permits pairing statistics with another result;
its type alone does not establish provenance or adapter completeness. See the
[composition guide](../../Sources/JSON/Documentation.docc/Composition.md).

## Composition completion accounting


The additive borrowed scope uses one original core ledger and the existing 49
metrics/default profiles. It does not introduce adjustable counters or widen any
cap. `chargeExternalWork` charges aggregate work before adapter work;
`reserveExternalLogicalBytes(bytes,count:overhead:)` checks multiplication then
addition and live retention before growth. Zero is valid but never establishes
that opaque caller storage is free. All reservations are conservative until scope
exit; repeated admission and physical aliases count again. Caller instructions,
allocator capacity and process RSS remain outside this contract.

`parseDocument` checks the full Data length against input and retained limits
before allocating its copied byte buffer, charges each copied byte, then invokes
the existing strict parser/source mapper. Caller Data is excluded. Input capacity
is per document: the preceding input count is checked and retired into the same
cumulative report before the next document. Work, retention, accepted-capacity
history and first-failure identity never restart. Remaining core counts are
conservative across the invocation; full source bytes/root/map and re-admitted
Pointer inputs/outputs remain charged. URI identity is core work/storage, never
schema compilation or compiledPlanBytes.

Completion admits an additional 424 logical bytes (32 header + 49 eight-byte
counter slots) and 50 work units (49 counter copies and one final work update).
All result reservations overlap this report buffer. Overflow, quota failure and
cancellation prevent a completed result/report even when caught by the adapter.
No report is emitted after an error. Native accounting-version-1 weights and the
ordinary entry points' accounting stay unchanged. This proof slice has no public
release/promote tokens, refunds or partial reports. Independent adapter events
and failing omitted-charge mutants qualify the authored adapter only.
39 changes: 39 additions & 0 deletions Documentation/CI/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
# Composition proof reconciled with S3

Reconciliation with main `c79d48c15f83ef88bed9459d9abcf05e54bd0bea` retains
all schema-example, Navigation and composition behavior. Exact combined roots
are 697 Mac / 681 Linux / 593 WASI functions, with 325 Mac/Linux and 288 WASI
parameter invocations. Shared-state counts remain 47 Mac / 46 Linux. Nine products,
eight library DocC catalogs, all 49 cap probes and all 126 restored identities
remain mandatory. Both focused composition lanes and the complete S3 consumer,
ownership, construction, matrix and corpus-lifetime checks are retained. The
15-case Python harness rejects omitted dispatch and stale integration counts.
Required check names and protections are unchanged. Quiet-host qualification
remains pending separately; no timing comparison is scheduled by this integration.

The following sections preserve their original source-specific counts.

# Combined S3 / composition / Navigation qualification

The current inventory requires 695 Mac / 679 Linux / 591 WASI functions,
Expand All @@ -14,6 +29,21 @@ The following sections preserve their original increment histories.

# Native CI

## Current composition proof

The additive composition proof retains current main's Navigation product and
adds two native boundary functions: exact discovery is 599 native / 583 Linux /
495 WASI functions, with the same 325 native/Linux and 288 WASI parameter cases.
Shared-state expectations remain 43 native / 42 Linux. Native/Linux still require
33/29 commands, all 126 restored identities and 49 cap probes. The separate
composition-proof lane additionally checks 26 public functions in both modes,
two rejected omitted-charge mutants and 14 normal compiler rejections. Known
compiler crashes remain separate limitations. Native CI now has five matrix
lanes including composition-proof; protected names/settings are unchanged.
The [controlled performance contract](../Qualification/Composition/Performance.md)
requires genuine pre/post idle admission on its isolated native runner. The
following counts retain their original stage context; receipts establish passes.

## Reconciled Navigation and composition increment

Current source retains composition from main and adds the opt-in Navigation
Expand Down Expand Up @@ -252,3 +282,12 @@ The focused composition lane also runs on the Linux architecture jobs and Native
Its separate-package discovery is additive; all root and 49-cap expectations remain intact.
The qualification harness now has 15 regression functions, retaining its original 14.
See [composition qualification](../Architecture/CompositionQualification.md) for finite capacity outcomes.

The additive `composition-proof` lane runs two native boundary functions and the
independent 26-function public suite in both configurations, two omitted-charge
mutants and 14 separate diagnostic builds. Both focused lanes run on Linux;
Native / composition-proof is separate from the protected names. The original
reference-load lane and its exact counts remain required. The controlled
composition performance job compares pinned main 30631e8 with this successor;
[the reconciliation record](../Qualification/Composition/Reconciliation.md) states
the expected accounting delta and unchanged idle/review thresholds.
70 changes: 70 additions & 0 deletions Documentation/Qualification/Composition/Performance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Composition quiet-host qualification

The Release comparison retains the full baseline
`30631e8fe60201518c00544c172340ec6576e0d5`, identical public benchmark source,
runtime/resource paths, two pinned inputs, five operations, ABBA ordering and
18 observations per revision/row. The review trigger remains a positive elapsed
increase greater than 10%. Every process must have three consecutive one-second
intervals at least 90% idle both before and immediately after execution. There
is no retry or settling period after a failed post window. Failed host admission
produces a blocked receipt, not a performance result or exception.

## Observer investigation

The three attempts of workflow `38810354356`, on composition proof head
`7ac75818f7974ab17d1bc44a39b08214eab15f16`, stopped at immediate post windows:
87.79%, 71.38% and 54.29% minimum idle. Those receipts remain immutable. They
show bursts of system CPU but contain no process attribution, so they cannot
establish which process caused every burst or quantify a regression.

The former observer starts `top -l 4 -n 0 -s 1` for each window. Limiting its
display does not avoid system-wide process inspection. A local ABBA diagnostic
found roughly 1.2 CPU seconds per four-snapshot invocation even with framework
and memory-map display disabled. This is diagnostic evidence on a busy Mac,
not a controlled performance result. It motivates checking observer overhead
on the actual qualification host rather than rerunning the old gate blindly.

`composition_host.py` samples the supported Darwin
`host_statistics(HOST_CPU_LOAD_INFO)` interface directly. SDK
`mach/host_info.h` and `mach/machine.h` define four unsigned 32-bit CPU tick
counters in user/system/idle/nice order. Four snapshots yield three one-second
interval deltas; boot totals are never treated as interval load. Checked shapes,
normal counter wrap, missing samples and nonfinite values have rejection tests.
All global activity, including the sampler's own CPU, remains in the denominator.
This removes process enumeration from admission without changing its threshold,
interval count or immediate-post policy. The owned Mach send right is released.

Each new native receipt includes a pre-measurement ABBA observer experiment:
the exact former `top` command twice and sampler-only windows twice. It records
independent CPU ticks, observer CPU time, elapsed time and raw top logs. Process
and thermal snapshots are outside measurement windows; a failed post window
additionally captures diagnostics without changing the failure. Paging between
the pre/post windows now also blocks a comparison. No workload is stopped.

Executable replacement, binary digest checks and a separately labelled preparation
launch precede pre-run admission for every process, equally for both revisions.
Preparation outputs are retained and checked for discovery/nonfinite values but
are never qualification samples. This lets first-launch cache/resource setup
settle before admission; it does not establish which service caused an old burst.
The measured processes still each perform their original two warmup sets and
nine samples. The first post-run native snapshot precedes JSON decoding, hashing
or receipt serialization. Eight harness tests reject omitted/changed workloads,
nonfinite/incomplete timings, malformed counters and sub-threshold intervals,
and preserve completion's non-additive peak-retention oracle. This Python
harness is distinct from the unchanged Swift Testing behavior suites.

## Interpretation

Unchanged native API counters must match. Composition completion intentionally
adds 50 work units and 424 final logical retained bytes; peak is
`max(previous peak, previous final retained + 424)`. Logical accounting does not
measure RSS. The pinned frozen schema resource is JSON/Pointer input here, not
protocol conformance or schema evaluation. Compiler ownership limitations and
bounded adapter-capacity refusals remain documented separately.

Run `python3 -B Scripts/qualify_composition_performance.py --output <new-directory>`
on the supported native host. Actual final-source outcomes, command exits,
source/fixture/binary hashes and materialized logs belong in the return evidence.
This contract does not claim an unexecuted pass. Marking the PR ready additionally
requires its current required native/safety/docs checks and relevant Linux and
archive/proof jobs; merge, tagging and publication remain separate actions.
Loading
Loading