diff --git a/research/partial-observation-boundaries/README.md b/research/partial-observation-boundaries/README.md new file mode 100644 index 0000000000..06692e5672 --- /dev/null +++ b/research/partial-observation-boundaries/README.md @@ -0,0 +1,234 @@ +# Exact dirty-boundary costs with partial output observations + +**Finite interface certificate; no new supplier or kappa.** In the model below, +observing only part of the output first lowers the optimal adapter cost at +three fixed calls. Among all 4,808 instances of lengths one through three, +100 have a strict saving over observing both outputs at every cut. The largest +gap is **one versus three paid XOR adapters**, with an exact dirty-state witness. +Every improving instance has one output with zero net final update. Among the +1,824 instances with nonzero final updates to both outputs, there is no saving +from weakening the observations in this domain. + +This answers a finite optimization question excluded by the earlier +[fully restored cut planner](../reader-restoration-cuts-seed53/README.md). +It does not repair an error in that planner. The result distinguishes an +observation boundary from a requirement to restore the entire state there. +The claim is in [discussion #2007](https://github.com/CrocSwap/integer-mult-bounds/discussions/2007). + +## Exact model and comparators + +There are four distinct binary coordinates `(d0,d1,w0,w1)`, all arbitrarily +dirty. A fixed child, indexed by `(o,s,t)` in `{0,1}^3`, performs in this order: + +``` +d[o] ^= w[s] +w[t] ^= w[1-t] +``` + +Equivalently, `C(B,G):(D,W) -> (D+B W, G W)`, where B is one of four elementary +two-by-two matrices and G is either elementary workspace shear. These two G's +do not commute. The eager reference immediately reverses each child's workspace +shear; its data increment after i children is `(sum_{j<=i} B_j) W_initial`. + +An adapter is exactly one of six scalar XORs: the two workspace shears, or +`d[o] ^= w[s]` for any o,s. Every adapter costs one, including entry preparation, +correction and final restoration. Arbitrary finite words of adapters are allowed +before the first child and after each child, but **never inside a child**. +There are no output-to-workspace gates, output-to-output gates, extra registers, +copies, erasures, free relabelings, or clean-input assumptions. + +After all adapters at each interior cut, a mask 1, 2 or 3 observes respectively +`d0`, `d1` or both. Only those rows must equal the eager reference at that cut. +The observer does not change the state or choose subsequent children. There are +no observers inside an adapter word. At the final cut, **all four coordinates** +must equal eager execution, including both incoming workspace values. +Future output rows may be prewritten while they are unobserved. + +The main comparison changes only these observation masks. The full-observation +optimizer has the same six adapters and may also carry noncanonical workspace; +it must present both correct data rows at every cut. A second, narrower comparator +enumerates every contiguous partition, with no entry adapters or output repair. +At each chosen cut it requires correct data and restores workspace by a shortest +word in the two workspace shears. It also checks any prescribed observations +inside an uncut interval. This is the exact two-by-two matrix analogue of a +fully restored partition plan, not a claim about the earlier diagonal fixture. + +Fixed children cost two XORs each. The complete cost **in this declared scalar +model** is therefore `2*n + minimum_adapter_cost`. Other gate alphabets and +ways to replace, split or skip a child are outside this optimum. + +## A sharp noncommuting three-call witness + +Choose children `(0,0,0), (1,0,1), (1,0,0)` and masks `(3,1,3)`. +Only the second cut hides `d1`. Insert the single adapter `w1 ^= w0` after +the first child. With arbitrary initial `(d0,d1,w0,w1)=(x,y,a,b)`: + +| Completed boundary | Candidate state | Eager data required there | +| --- | --- | --- | +| Child 1 and adapter | `(x+a, y, a+b, a)` | both: `(x+a,y)` | +| Child 2 | `(x+a, y+a+b, a+b, b)` | only `d0=x+a` | +| Child 3 | `(x+a, y, a, b)` | all four coordinates | + +All sums are in characteristic two. This works on arbitrary dirty bit streams +coordinatewise, without allocating a temporary. The eager reference uses three +restoring XORs, and even the more general full-observation optimizer needs three. +The complete fixed-call bills are seven and nine XORs: fourteen versus eighteen +scalar data reads and seven versus nine writes, with the same four stored scalars. +The single adapter can also be placed after child 2; these are the only two +one-adapter words in this fixture. It is not an eager return at either location. + +Here is a direct proof of the sharp costs, separate from the census. Write U +for `w0 ^= w1` and L for `w1 ^= w0`. With no adapter, workspace ends at ULU, +not I, so at least one adapter is necessary. The displayed word attains one. +Each workspace shear is an odd permutation of the three nonzero binary vectors. +The three child shears have odd product. Thus any restoring program uses an +odd number of workspace adapters. A program with at most two adapters must use +exactly one workspace adapter, and at most one data adapter. Checking its four +possible boundary positions, the workspace adapter must be L between child 1 +and child 2, or between child 2 and child 3. Both choices already give the correct +final data, but their second-cut `d1` differs from eager by b. A single data +adapter adds a nonzero row of the current invertible workspace matrix to one +output; that contribution persists to the final output because no operation +reads D into workspace or mixes outputs. It cannot fix the intermediate error +while retaining the final endpoint. Hence full observation needs at least three +adapters; eager execution attains three. Literal enumeration rejects all 385 +boundary words of length at most two under full observation. + +The unchanged final `d1` is material, not hidden. If these toy children may be +opened or replaced, their overall endpoint is simply `d0 ^= w0`; the fixed-call +bill is not an unrestricted circuit optimum or a new multiplication algorithm. + +## Why the finite search proves the stated minima + +Every permitted prefix has the unique form + +``` +W = P W_initial, D = D_initial + Q W_initial, +P in GL_2(F2), Q in Mat_2(F2). +``` + +There are exactly `6*16=96` states. The representation is faithful on all dirty +inputs: the full four-by-four matrix is `[[I,Q],[0,P]]`. Each of the six adapter +edges updates exactly this matrix. Every allowable word is a path in this graph, +and every path is an allowed word. Nonnegative unit costs mean cycles can be +removed, so no length cutoff on adapter words is needed. + +Starting at `(I,0)`, the primary solver computes a shortest adapter closure, +applies the next fixed child, computes another closure, and removes states with +incorrect observed rows. Induction on the child index shows its labels are +exactly the minimum costs for all admissible prefixes. Requiring `(I,sum B_i)` +at the final cut proves the claimed minimum and supplies an attaining word. + +A separately written engine instead uses literal four-by-four bit matrices and +a chronological layered graph. Adapter edges cost one; mandatory child edges +cost zero after subtracting the common fixed-child bill. It checks observations +before advancing to the next child and runs 0-1 BFS. It does not use the primary +state encoding, generator table, shortest-word routine or optimizer. Both +engines agree on every instance. A second literal enumeration independently +checks every restored-partition price. All 4,808 primary attaining words are +replayed on all 16 dirty Boolean inputs, for **76,928 complete executions**. + +The instance count is +`8 + 8^2*3 + 8^3*3^2 = 4,808`. +There is no quotienting by symmetry, seed selection, random sampling, tolerance +or floating arithmetic. The complete cost census is: + +| Children | Partial min | Full-observation min | Partition min | Instances | +| ---: | ---: | ---: | ---: | ---: | +| 1 | 1 | 1 | 1 | 8 | +| 2 | 0 | 0 | 0 | 48 | +| 2 | 1 | 1 | 2 | 48 | +| 2 | 2 | 2 | 2 | 96 | +| 3 | 1 | 1 | 1 | 2,016 | +| 3 | 1 | 1 | 3 | 288 | +| 3 | 1 | 2 | 3 | 52 | +| 3 | 1 | 3 | 3 | 48 | +| 3 | 2 | 2 | 3 | 1,964 | +| 3 | 3 | 3 | 3 | 240 | + +Thus 100 instances improve specifically because of partial observations, of +which 48 use both noncommuting workspace shears. There are separately 2,400 +improvements over the restricted partition comparator. Those larger counts +must not be attributed solely to weakening observations. + +[RESULT.json](RESULT.json) records these counts, the witness, a canonical hash +of every cost row, a separate hash of all attaining words, source checks and +controls. Hashes compact the receipts; the verifier recomputes the mathematics. +Ten failure controls address omitted/wrong/moved adapters, newly observed output, +unpaid work, data-correct but workspace-wrong endpoints, changed readers and +invalid declared boundaries. The zero-workspace false positive demonstrates why +clean-only testing would miss an error. All checks remain enabled with `-O`. + +## Reproduce + +Python 3.12, standard library only, from the repository root: + +```sh +python3 -I -O -B research/partial-observation-boundaries/verify.py +python3 scripts/research_package.py --package research/partial-observation-boundaries --check-only +python3 scripts/research_package.py --package research/partial-observation-boundaries +``` + +Expected main line: + +``` +PASS_PARTIAL_OBSERVATION_BOUNDARIES instances=4808 states=96 partial_gains=100 productive_gains=0 dirty_inputs=76928 controls=10 no_new_kappa +``` + +Normal replay is read-only. `--write-result NEW_PATH` creates a fresh receipt +exclusively; it does not overwrite one. The registered replay also authenticates +historical catalogue/selected-record provenance and every declared live source. +Four package-integrity tests exercise real source/receipt corruption, refusal +to overwrite an existing result, and optimized read-only relocation from an +empty working directory. + +## Scope, provenance and credit + +Base: `1ff9c501b624ec354e1330b1b9be571a6adba9cb`. Chooser nonce: +`86e84927a2ed518696c231e449d50f59`; revision 3; recursive-interfaces; +`persistent-boundary`, scope key `73c0a233e7e1abdccbfe`. [TASK.json](TASK.json) +declares a **changed** scope: this is a finite abstract replacement interface, +not a selected-supplier call pair. It prices a precise checkpoint relaxation +with noncommuting workspace maps, rather than asserting a recursive saving. +[SOURCES.json](SOURCES.json) pins the predecessor notes by base, Git blob and +SHA256; no predecessor solver is imported or frozen package modified. + +Credit Aaron Wang/Navinor's PR1155 for restoration coalescing, Adam Gall's +`observer-safe-restoration` for the exact terminal/prefix defect criterion, and +Sagar Thakrar's `reader-restoration-cuts-seed53` for the fully restored interval +planner. These remain valid in their declared domains. The extra contribution +here is the observation-constrained 96-state optimum, its complete small-domain +census, sharp witness and independently implemented checks. No priority is +claimed for linear cancellation, shortest paths or dynamic programming. + +Nearby PR1930 (Cinna Davis), head `11cb2266f7e99279c440a301ea4b862587a9afa2`, +uses output-to-source prefix chains, a different contract. PR1155's contextual +head at entry was `d7d50406fbdc7ac4141ba454b2e2440dc74942db`; neither pending PR +is an implementation dependency. This account's PR1414/1491/1495/1496 concern +paid coordinate-address calls in formal group algebras; the present gates are +scalar boundary XORs and those address-call results are not repeated. + +No source-level selected caller, recursive rank profile, physical instruction +dispatch or routing tariff, hosting, precision/tape, contracting recurrence, +selected cold replay, Lean theorem or multiplication-exponent improvement is +established. A bitstream implementation multiplies payload XOR/read/write counts +by its length but still must pay and validate control, transport and hosting. +No complex kappa claim is made, so the incumbent packing-cap calculator is not +being applied to this different interface model. + +Prepared for commdata2338 (@commdata2338), with substantial OpenAI Codex assistance. +New work is Apache-2.0. Source attribution and licenses are retained. Independent +implementation checks are not independent human review. + +## Open next questions + +- Does a specified admitted supplier contain a mandatory call chain with this + observation pattern and noncanonical workspace contract? Map an actual pair + or triple of complete call boundaries to the model and charge every adapter, + owner, frame and transport obligation. The missing source-bound caller and + permission to hide the intermediate output are the blockers to adoption. +- At four calls, can partial observations lower cost when both final output + update rows are nonzero? The same 96-state solvers can exhaust the 110,592 + length-four instances. This would test the exact boundary of the negative + three-call census; it is outside the present registered domain, and a positive + toy result would still need the physical binding above. diff --git a/research/partial-observation-boundaries/RESULT.json b/research/partial-observation-boundaries/RESULT.json new file mode 100644 index 0000000000..c4c6d03c87 --- /dev/null +++ b/research/partial-observation-boundaries/RESULT.json @@ -0,0 +1,164 @@ +{ + "schema_version": 1, + "status": "PASS_PARTIAL_OBSERVATION_BOUNDARIES", + "scope": "exact finite scalar XOR boundary model; no supplier or kappa claim", + "states": 96, + "generator_edges": 576, + "instances": 4808, + "instances_by_length": { + "1": 8, + "2": 192, + "3": 4608 + }, + "histogram_columns": [ + "children", + "partial_adapter_min", + "full_observation_adapter_min", + "restored_partition_min", + "instances" + ], + "histogram": [ + [ + 1, + 1, + 1, + 1, + 8 + ], + [ + 2, + 0, + 0, + 0, + 48 + ], + [ + 2, + 1, + 1, + 2, + 48 + ], + [ + 2, + 2, + 2, + 2, + 96 + ], + [ + 3, + 1, + 1, + 1, + 2016 + ], + [ + 3, + 1, + 1, + 3, + 288 + ], + [ + 3, + 1, + 2, + 3, + 52 + ], + [ + 3, + 1, + 3, + 3, + 48 + ], + [ + 3, + 2, + 2, + 3, + 1964 + ], + [ + 3, + 3, + 3, + 3, + 240 + ] + ], + "partial_observation_improvements": 100, + "noncommuting_partial_improvements": 48, + "improvements_over_restored_partition": 2400, + "both_outputs_nonzero_cases": 1824, + "both_outputs_nonzero_partial_improvements": 0, + "literal_dirty_input_replays": 76928, + "census_sha256": "5dfeaea1efecf76eff484a2e875d4caad57106e08e67ff3ab0fb3e389e2d12dc", + "attaining_words_sha256": "064de2bb71f8469b146ee126638d276f9764ee263dc310a8cf732ed565091262", + "independent_search": "0-1 BFS over full 4x4 binary maps and chronological layers, all 4808 instances", + "witness": { + "children": [ + [ + 0, + 0, + 0 + ], + [ + 1, + 0, + 1 + ], + [ + 1, + 0, + 0 + ] + ], + "masks": [ + 3, + 1, + 3 + ], + "adapter_words": [ + [], + [ + 1 + ], + [], + [] + ], + "adapter_cost": 1, + "fixed_child_xors": 6, + "total_xors": 7, + "full_observation_min_total_xors": 9, + "restored_partition_min_total_xors": 9 + }, + "source_inputs_checked": [ + "research/observer-safe-restoration/README.md", + "research/reader-restoration-cuts-seed53/NOTICE.md", + "research/reader-restoration-cuts-seed53/README.md" + ], + "negative_controls": [ + "omit_noncanonical_adapter", + "use_eager_inverse_instead", + "observe_hidden_second_output", + "move_adapter_before_first_child", + "omit_adapter_charge", + "data_correct_workspace_wrong", + "change_reader_source", + "missing_final_observation", + "missing_cut", + "unknown_child_shear" + ], + "positive_controls": [ + "attaining_dirty_witness", + "clean_only_false_positive", + "six_adapter_involutions", + "noncommuting_workspace_shears", + "independent_full_observation_floor" + ], + "full_observation_short_words_rejected": 385, + "partial_one_adapter_words": 2 +} diff --git a/research/partial-observation-boundaries/SOURCES.json b/research/partial-observation-boundaries/SOURCES.json new file mode 100644 index 0000000000..15be4dff1d --- /dev/null +++ b/research/partial-observation-boundaries/SOURCES.json @@ -0,0 +1,36 @@ +{ + "schema_version": 1, + "base_commit": "1ff9c501b624ec354e1330b1b9be571a6adba9cb", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "ae6a99f28aa468649eb66a361e764d769363bf11", + "sha256": "10b9e340fd8e354a37a0c4813c90e5de1dd42044b5f6aca4a3fc5a42b685fa32" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "06c96f5579bb67ed1b86d334e78fbed7da6945f1", + "sha256": "6686ad58f1fbdbc87134a78901b60fd53a20d3984355e91defbbebc9e9fa8075" + }, + { + "path": "research/observer-safe-restoration/README.md", + "usage": "input", + "git_blob": "7ed4d89d17c5cdeba0e5e5eb160919e0e9c5a076", + "sha256": "fc672e69f4b3363fa7736efa4b4103c755fcb72f6f5d45e5d268c10e133a914d" + }, + { + "path": "research/reader-restoration-cuts-seed53/NOTICE.md", + "usage": "input", + "git_blob": "c86cefe681b8307a09ea4235b2421fba923f229e", + "sha256": "0a795d70b5c0e74b5cfbe3f1302a7eac7bc057748bbc2cc48d751b5470e758c8" + }, + { + "path": "research/reader-restoration-cuts-seed53/README.md", + "usage": "input", + "git_blob": "f58d1581de51da4aa369a354bd19a9962144eac9", + "sha256": "a073617e65be95d8fe7ceba41387d85171950d19f5e03f2ad1d9d1c8a6f4dae7" + } + ] +} diff --git a/research/partial-observation-boundaries/TASK.json b/research/partial-observation-boundaries/TASK.json new file mode 100644 index 0000000000..afd6e437c2 --- /dev/null +++ b/research/partial-observation-boundaries/TASK.json @@ -0,0 +1,70 @@ +{ + "schema_version": 1, + "task_id": "persistent-boundary", + "priority_revision": 3, + "base_commit": "1ff9c501b624ec354e1330b1b9be571a6adba9cb", + "original_card": { + "id": "persistent-boundary", + "question": "Can a specified pair of consecutive calls eliminate a restoration that the next call reconstructs?", + "starting_points": [ + "research/heterogeneous-owner-assembly/README.md", + "certificates/selected-result.json" + ], + "methods": [ + "Write typed entry/exit contracts and a contracting multitype recurrence.", + "Derive a lower bound for the compulsory transitions of one proposed typed interface." + ], + "acceptance_gate": "Show actual eliminated work and legal composition; changing a scalar recurrence into a matrix is insufficient.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Find a concrete consecutive-call boundary in the selected review and list its entry/exit obligations.", + "scope": "Positive control / source-bound orientation only; this is not a new construction or full cold replay." + }, + "first_experiment": "Specify two call types and one operation actually removed by composition; derive the normalized recurrence.", + "bottleneck_link": "Eliminating compulsory boundary work can lower recursive cost beyond packing gains.", + "allowed_variations": [ + "A two-type pilot or one explicit deferred restoration." + ], + "scope_change_triggers": [ + "Rewriting the old scalar recurrence as a matrix without eliminating work." + ], + "overlap_terms": [ + "persistent boundary", + "deferred restoration", + "multitype" + ] + }, + "scope_status": "changed", + "actual_scope": "Exact minimum paid scalar XOR adapters for 4808 fixed two-output/two-workspace binary call chains of lengths 1..3, with partial output observations and arbitrary dirty inputs.", + "scope_change_reason": "A finite abstract replacement interface is studied instead of asserting a selected-supplier call pair. Intermediate observations may leave workspace and unobserved output rows noncanonical; the earlier fully restored diagonal partition model excludes this state space.", + "bottleneck_link": "Scoped interface side result: exact adapter tariffs show when partial observations eliminate boundary work. No selected recursive profile or kappa improvement follows without a real caller and fully charged physical binding.", + "budget_minutes": 60, + "stop_rule": "Exhaust exactly lengths 1..3 and the declared six-adapter/eight-child alphabet; do not expand the domain or infer native admission. Reserve final quarter of a 60-minute session for checks and handoff.", + "outcome": "completed", + "evidence": "RESULT.json: two independent exact searches agree on 4808 instances; 100 partial-observation improvements, none of the 1824 instances with both update rows nonzero; 76928 literal dirty-input runs and 10 failure controls. README supplies the finite-state completeness proof and direct sharp 1-versus-3 proof.", + "author": "commdata2338", + "ai_assistance": "Substantial OpenAI Codex assistance in scoping, mathematics, implementation, verification and writing; no independent human or proof-assistant review.", + "open_questions": [ + { + "question": "Does a named admitted supplier expose the partial-observation/noncanonical-workspace contract?", + "why_it_matters": "Only a source-bound caller with all physical charges could turn the finite boundary saving into an admitted recursive improvement.", + "first_step": "Extract one actual call pair or triple and its allowed observation cuts; verify every dirty row and price adapters, frames, ownership, control and transport.", + "blocked_by": "No admitted selected caller or complete physical tariff has been identified." + }, + { + "question": "At exactly four calls, can partial observations save adapters when both final output-update rows are nonzero?", + "why_it_matters": "Tests the boundary of the exhaustive negative result for lengths at most three.", + "first_step": "Run both exact solvers on the 110592 length-four instances with the unchanged alphabet and compare all full-observation minima.", + "blocked_by": "Outside the declared finite domain; even a positive answer would remain conditional on a physical caller." + } + ], + "seed": "86e84927a2ed518696c231e449d50f59", + "track": "recursive-interfaces", + "scope_key": "73c0a233e7e1abdccbfe", + "discussion": "https://github.com/CrocSwap/integer-mult-bounds/discussions/2007" +} diff --git a/research/partial-observation-boundaries/VERIFY.json b/research/partial-observation-boundaries/VERIFY.json new file mode 100644 index 0000000000..9f4b46c907 --- /dev/null +++ b/research/partial-observation-boundaries/VERIFY.json @@ -0,0 +1,33 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [ + "research/observer-safe-restoration/README.md", + "research/reader-restoration-cuts-seed53/NOTICE.md", + "research/reader-restoration-cuts-seed53/README.md" + ], + "commands": [ + { + "argv": [ + "{python}", + "-I", + "-O", + "-B", + "{package}/verify.py" + ], + "timeout_seconds": 120, + "checks": "Both exact minimum searches on all 4808 instances; independent partition prices; all attaining words on 16 arbitrary dirty inputs; 385 short-word exclusions; ten negative controls; source and full receipt equality." + }, + { + "argv": [ + "{python}", + "-B", + "{package}/test_package.py" + ], + "timeout_seconds": 120, + "checks": "Actual source/receipt corruption rejection, exclusive output preservation, optimized read-only relocation from an empty working directory." + } + ] +} diff --git a/research/partial-observation-boundaries/model.py b/research/partial-observation-boundaries/model.py new file mode 100644 index 0000000000..06fd8289c2 --- /dev/null +++ b/research/partial-observation-boundaries/model.py @@ -0,0 +1,166 @@ +"""Exact six-generator boundary model; standard library only, Apache-2.0. + +Prepared for commdata2338 with substantial OpenAI Codex assistance. +Rows are two-bit coefficient vectors on the incoming workspace. No predecessor +implementation is imported. Fixed children are never split by adapters. +""" +from collections import deque +from functools import lru_cache +from heapq import heapify, heappop, heappush +from itertools import product + +INF = 10**6 +FRAMES = tuple((a, b) for a in range(1, 4) for b in range(1, 4) if a != b) +STATES = tuple((a, b, c, d) for a, b in FRAMES for c, d in product(range(4), repeat=2)) +INDEX = {s: i for i, s in enumerate(STATES)} +START = INDEX[(1, 2, 0, 0)] +GATES = ("w0^=w1", "w1^=w0", "d0^=w0", "d0^=w1", "d1^=w0", "d1^=w1") +CHILDREN = tuple(product(range(2), repeat=3)) # output, reader, shear destination + + +def gate(state, op): + rows = list(state) + if op < 2: + rows[op] ^= rows[1 - op] + else: + dest, source = divmod(op - 2, 2) + rows[2 + dest] ^= rows[source] + return tuple(rows) + + +EDGES = tuple(tuple(INDEX[gate(s, g)] for g in range(6)) for s in STATES) + + +def child(state, spec): + dest, source, shear = spec + return gate(gate(state, 2 + 2 * dest + source), shear) + + +def reference_increment(q, spec): + rows = list(q) + rows[spec[0]] ^= 1 << spec[1] + return tuple(rows) + + +def observed(state, q, mask): + return all(not (mask & (1 << j)) or state[2 + j] == q[j] for j in range(2)) + + +def closure(initial): + """Multi-source Dijkstra; adapters have unit cost, initial labels may differ.""" + costs = list(initial) + origins = list(range(96)) + heap = [(value, i) for i, value in enumerate(costs) if value < INF] + heapify(heap) + while heap: + value, s = heappop(heap) + if value != costs[s]: + continue + for t in EDGES[s]: + if value + 1 < costs[t]: + costs[t] = value + 1 + origins[t] = origins[s] + heappush(heap, (value + 1, t)) + return tuple(costs), tuple(origins) + + +@lru_cache(maxsize=None) +def shortest_word(source, target, workspace_only=False): + queue = deque([source]) + previous = {source: None} + while queue: + s = queue.popleft() + if s == target: + result = [] + while previous[s] is not None: + s, op = previous[s] + result.append(op) + return tuple(reversed(result)) + for op, t in enumerate(EDGES[s][:2] if workspace_only else EDGES[s]): + if t not in previous: + previous[t] = (s, op) + queue.append(t) + raise ValueError("unreachable boundary") + + +def validate_instance(calls, masks): + if not calls or len(calls) != len(masks) or masks[-1] != 3: + raise ValueError("nonempty calls with matching masks and full final observation required") + if any(tuple(c) not in CHILDREN for c in calls) or any(type(m) is not int or m not in (1, 2, 3) for m in masks): + raise ValueError("outside the declared child/observation alphabet") + + +def solve(calls, masks, witness=False, canonical=False): + """All boundary words, including entry preparation and final restoration.""" + validate_instance(calls, masks) + initial = [INF] * 96 + initial[START] = 0 + costs, _ = closure(initial) + history = [] + q = (0, 0) + for spec, mask in zip(calls, masks): + permutation = tuple(INDEX[child(s, spec)] for s in STATES) + inverse = {t: s for s, t in enumerate(permutation)} + raw = [INF] * 96 + for s, t in enumerate(permutation): + raw[t] = costs[s] + costs, origins = closure(raw) + q = reference_increment(q, spec) + costs = tuple(v if observed(s, q, mask) and (not canonical or s[:2] == (1, 2)) else INF + for s, v in zip(STATES, costs)) + if witness: + history.append((origins, inverse)) + target = INDEX[(1, 2, *q)] + value = costs[target] + if value >= INF: + raise ValueError("unreachable requested endpoint") + if not witness: + return value + words = [] + for origins, inverse in reversed(history): + source = origins[target] + words.append(shortest_word(source, target)) + target = inverse[source] + words.append(shortest_word(START, target)) + return value, tuple(reversed(words)) + + +def restored_partition(calls, masks): + """No entry adapters or output repair; only full workspace returns at cuts. + + Every chosen cut must have canonical data too. Observations inside an + interval are checked even when that boundary is not a chosen cut. + """ + validate_instance(calls, masks) + best = None + n = len(calls) + for bits in range(1 << (n - 1)): + cuts = tuple(i for i in range(n) if i == n - 1 or bits & (1 << i)) + state, q, price, valid = STATES[START], (0, 0), 0, True + for i, (spec, mask) in enumerate(zip(calls, masks)): + state = child(state, spec) + q = reference_increment(q, spec) + if i in cuts: + if state[2:] != q: + valid = False + break + target = (1, 2, *q) + price += len(shortest_word(INDEX[state], INDEX[target], True)) + state = target + if not observed(state, q, mask): + valid = False + break + if valid: + record = (price, tuple(i + 1 for i in cuts)) + if best is None or record < best: + best = record + if best is None: + raise RuntimeError("the eager partition must be feasible") + return best + + +def instances(): + for n in range(1, 4): + for calls in product(CHILDREN, repeat=n): + for prefix in product((1, 2, 3), repeat=n - 1): + yield calls, prefix + (3,) diff --git a/research/partial-observation-boundaries/test_package.py b/research/partial-observation-boundaries/test_package.py new file mode 100644 index 0000000000..da2f34d8af --- /dev/null +++ b/research/partial-observation-boundaries/test_package.py @@ -0,0 +1,91 @@ +"""Integrity and portable-replay tests, using only isolated temporary copies. + +Prepared for commdata2338 with substantial OpenAI Codex assistance. Apache-2.0. +""" +from contextlib import contextmanager +from hashlib import sha256 +import json +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import unittest + +PACKAGE = Path(__file__).resolve().parent +ROOT = PACKAGE.parents[1] + + +@contextmanager +def export(): + with tempfile.TemporaryDirectory(prefix="partial-observation-test-") as name: + root = Path(name) + package = root / "research" / PACKAGE.name + shutil.copytree(PACKAGE, package, ignore=shutil.ignore_patterns("__pycache__")) + for item in json.loads((PACKAGE / "SOURCES.json").read_text())["files"]: + if item["usage"] == "input": + target = root / item["path"] + target.parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(ROOT / item["path"], target) + yield root, package + + +def run(package, *args, cwd=None): + return subprocess.run([sys.executable, "-I", "-O", "-B", str(package / "verify.py"), *map(str, args)], + cwd=cwd, capture_output=True, text=True, timeout=120, check=False) + + +def hashes(root): + return {str(p.relative_to(root)): sha256(p.read_bytes()).hexdigest() + for p in root.rglob("*") if p.is_file()} + + +class PackageIntegrity(unittest.TestCase): + def test_source_corruption_rejects(self): + with export() as (root, package): + item = next(x for x in json.loads((package / "SOURCES.json").read_text())["files"] if x["usage"] == "input") + target = root / item["path"] + target.write_bytes(target.read_bytes() + b"\ncorrupt source control\n") + result = run(package) + self.assertNotEqual(result.returncode, 0) + self.assertIn("source drift", result.stderr) + + def test_receipt_corruption_rejects(self): + with export() as (_, package): + target = package / "RESULT.json" + data = json.loads(target.read_text()) + data["partial_observation_improvements"] += 1 + target.write_text(json.dumps(data)) + result = run(package) + self.assertNotEqual(result.returncode, 0) + self.assertIn("receipt mismatch", result.stderr) + + def test_existing_result_is_preserved(self): + with export() as (_, package): + target = package / "RESULT.json" + before = target.read_bytes() + result = run(package, "--write-result", target) + self.assertNotEqual(result.returncode, 0) + self.assertIn("refusing to overwrite", result.stderr) + self.assertEqual(target.read_bytes(), before) + + def test_readonly_relocation_from_empty_cwd(self): + with export() as (root, package): + empty = root / "empty" + empty.mkdir() + before = hashes(root) + paths = list(root.rglob("*")) + try: + for path in paths: + path.chmod(0o555 if path.is_dir() else 0o444) + result = run(package, cwd=empty) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertIn("PASS_PARTIAL_OBSERVATION_BOUNDARIES", result.stdout) + self.assertEqual(hashes(root), before) + finally: + for path in paths: + path.chmod(0o755 if path.is_dir() else 0o644) + + +if __name__ == "__main__": + unittest.main() diff --git a/research/partial-observation-boundaries/verify.py b/research/partial-observation-boundaries/verify.py new file mode 100644 index 0000000000..2ef5401f76 --- /dev/null +++ b/research/partial-observation-boundaries/verify.py @@ -0,0 +1,314 @@ +#!/usr/bin/env python3 +"""Independent literal search and arbitrary-dirty checks of the finite boundary model. + +Prepared for commdata2338 with substantial OpenAI Codex assistance. Apache-2.0. +Normal execution is read-only. --write-result requires a nonexistent output. +No assert statements are used, so optimized Python preserves every check. +""" +import argparse +from collections import Counter, deque +from functools import lru_cache +from hashlib import sha256 +from itertools import product +import json +from pathlib import Path +import sys + +PACKAGE = Path(__file__).resolve().parent +sys.path.insert(0, str(PACKAGE)) +import model + +WIRES = ((2, 3), (3, 2), (0, 2), (0, 3), (1, 2), (1, 3)) +IDENTITY = (1, 2, 4, 8) + + +def require(condition, message): + if not condition: + raise ValueError(message) + + +def xor(rows, destination, source): + result = list(rows) + result[destination] ^= result[source] + return tuple(result) + + +def apply_adapter(rows, op): + return xor(rows, *WIRES[op]) + + +def apply_child(rows, spec): + output, reader, drift = spec + rows = xor(rows, output, 2 + reader) + return xor(rows, 2 + drift, 3 - drift) + + +def eager_boundaries(calls, initial=IDENTITY): + rows = initial + result = [rows] + for spec in calls: + rows = apply_child(rows, spec) + rows = xor(rows, 2 + spec[2], 3 - spec[2]) + result.append(rows) + return tuple(result) + + +def literal_observed(rows, expected, mask): + return all(not (mask & (1 << j)) or rows[j] == expected[j] for j in range(2)) + + +@lru_cache(maxsize=None) +def independent_minimum(calls, masks): + """0-1 BFS on chronological full four-by-four Boolean linear maps. + + Uses neither the 96-state encoding, its edges, nor its optimizer. A child + edge costs zero because the same fixed child bill is paid by every plan. + An observation is checked before leaving its layer for the next child. + """ + eager = eager_boundaries(calls) + start = (0, IDENTITY) + target = (len(calls), eager[-1]) + distances = {start: 0} + queue = deque([(start, 0)]) + while queue: + node, price = queue.popleft() + if distances[node] != price: + continue + if node == target: + return price + layer, rows = node + if layer < len(calls) and (layer == 0 or literal_observed(rows, eager[layer], masks[layer - 1])): + next_node = (layer + 1, apply_child(rows, calls[layer])) + if price < distances.get(next_node, 10**9): + distances[next_node] = price + queue.appendleft((next_node, price)) + for destination, source in WIRES: + next_node = (layer, xor(rows, destination, source)) + if price + 1 < distances.get(next_node, 10**9): + distances[next_node] = price + 1 + queue.append((next_node, price + 1)) + raise ValueError("independent search failed to reach the eager endpoint") + + +def check_word(calls, masks, words, cost=None, inputs=None): + require(len(words) == len(calls) + 1, "wrong boundary word inventory") + require(all(type(op) is int and 0 <= op < 6 for word in words for op in word), "bad adapter") + actual_cost = sum(map(len, words)) + if cost is not None: + require(actual_cost == cost, "undercharged adapter word") + cases = tuple(product((0, 1), repeat=4)) if inputs is None else tuple(inputs) + for initial in cases: + expected = eager_boundaries(calls, initial) + rows = initial + for op in words[0]: + rows = apply_adapter(rows, op) + for i, spec in enumerate(calls): + rows = apply_child(rows, spec) + for op in words[i + 1]: + rows = apply_adapter(rows, op) + require(literal_observed(rows, expected[i + 1], masks[i]), "observed output mismatch") + require(rows == expected[-1], "final dirty-state mismatch") + return len(cases) + + +def independent_partition(calls, masks): + """Enumerate literal returns and partitions without the primary state graph.""" + eager = eager_boundaries(calls) + best = 10**9 + for optional in product((False, True), repeat=len(calls) - 1): + cuts = optional + (True,) + rows, cost = IDENTITY, 0 + for i, spec in enumerate(calls): + rows = apply_child(rows, spec) + if cuts[i]: + if rows[:2] != eager[i + 1][:2]: + break + found = False + for length in range(4): + for ops in product((0, 1), repeat=length): + trial = rows + for op in ops: + trial = apply_adapter(trial, op) + if trial == eager[i + 1]: + rows, cost, found = trial, cost + length, True + break + if found: + break + require(found, "workspace return exceeds the GL2 diameter") + if not literal_observed(rows, eager[i + 1], masks[i]): + break + else: + best = min(best, cost) + require(best < 10**9, "independent eager partition unreachable") + return best + + +def short_word_floor(calls, masks): + """All 385 boundary words with at most two adapters on four boundaries.""" + checked, partial_unit = 0, 0 + for length in range(3): + for locations in product(range(4), repeat=length): + if tuple(sorted(locations)) != locations: + continue + for ops in product(range(6), repeat=length): + words = tuple(tuple(op for where, op in zip(locations, ops) if where == j) for j in range(4)) + try: + check_word(calls, (3, 3, 3), words) + except ValueError: + checked += 1 + else: + raise ValueError("full-observer floor has a short counterexample") + if length == 1: + try: + check_word(calls, masks, words) + except ValueError: + pass + else: + partial_unit += 1 + require(checked == 385 and partial_unit == 2, "short-word coverage drift") + return {"full_observation_short_words_rejected": checked, "partial_one_adapter_words": partial_unit} + + +def source_check(): + sources = json.loads((PACKAGE / "SOURCES.json").read_text()) + root = PACKAGE.parents[1] + checked = [] + for source in sources["files"]: + if source["usage"] != "input": + continue + payload = (root / source["path"]).read_bytes() + require(sha256(payload).hexdigest() == source["sha256"], "source drift: " + source["path"]) + checked.append(source["path"]) + return checked + + +def expect_failure(action, label): + try: + action() + except ValueError: + return label + raise ValueError("negative control accepted: " + label) + + +def controls(): + calls = ((0, 0, 0), (1, 0, 1), (1, 0, 0)) + masks = (3, 1, 3) + price, words = model.solve(calls, masks, True) + require(price == 1 and words == ((), (1,), (), ()), "named witness drift") + check_word(calls, masks, words, price) + failures = [] + failures.append(expect_failure(lambda: check_word(calls, masks, ((), (), (), ())), "omit_noncanonical_adapter")) + failures.append(expect_failure(lambda: check_word(calls, masks, ((), (0,), (), ())), "use_eager_inverse_instead")) + failures.append(expect_failure(lambda: check_word(calls, (3, 3, 3), words), "observe_hidden_second_output")) + failures.append(expect_failure(lambda: check_word(calls, masks, ((1,), (), (), ())), "move_adapter_before_first_child")) + failures.append(expect_failure(lambda: check_word(calls, masks, words, 0), "omit_adapter_charge")) + failures.append(expect_failure(lambda: check_word(calls, masks, ((), (1,), (), (0,))), "data_correct_workspace_wrong")) + check_word(calls, masks, ((), (), (), ()), inputs=((0, 0, 0, 0), (1, 1, 0, 0))) + changed_reader = (calls[0], (1, 1, 1), calls[2]) + failures.append(expect_failure(lambda: check_word(changed_reader, masks, words), "change_reader_source")) + failures.append(expect_failure(lambda: model.solve(calls, (3, 1, 0)), "missing_final_observation")) + failures.append(expect_failure(lambda: model.solve(calls, (3, 1)), "missing_cut")) + failures.append(expect_failure(lambda: model.solve(((0, 0, 2),), (3,)), "unknown_child_shear")) + for op in range(6): + require(apply_adapter(apply_adapter(IDENTITY, op), op) == IDENTITY, "adapter involution") + require(apply_adapter(apply_adapter(IDENTITY, 0), 1) != apply_adapter(apply_adapter(IDENTITY, 1), 0), "drifts commute") + require(independent_minimum(calls, (3, 3, 3)) == 3, "full-observer lower bound") + return {"negative_controls": failures, "positive_controls": ["attaining_dirty_witness", "clean_only_false_positive", "six_adapter_involutions", "noncommuting_workspace_shears", "independent_full_observation_floor"]} + + +def certify(): + checked_sources = source_check() + require(len(model.STATES) == 96 and len(set(model.STATES)) == 96, "state coverage") + literal_maps = set() + for p0, p1, q0, q1 in model.STATES: + literal_maps.add((1 | q0 << 2, 2 | q1 << 2, p0 << 2, p1 << 2)) + require(len(literal_maps) == 96, "noninjective full dirty-map representation") + for s in model.STATES: + rows = (1 | s[2] << 2, 2 | s[3] << 2, s[0] << 2, s[1] << 2) + for op in range(6): + t = model.gate(s, op) + require(apply_adapter(rows, op) == (1 | t[2] << 2, 2 | t[3] << 2, t[0] << 2, t[1] << 2), "generator semantics") + hist = Counter() + per_length = Counter() + full_costs = {} + productive_cases = productive_improvements = partial_improvements = 0 + noncommuting_improvements = partition_improvements = dirty_checks = 0 + digest, witness_digest = sha256(), sha256() + for calls, masks in model.instances(): + n = len(calls) + cost, words = model.solve(calls, masks, True) + require(cost == independent_minimum(calls, masks), "independent shortest-path disagreement") + if calls not in full_costs: + full_costs[calls] = model.solve(calls, (3,) * n) + require(full_costs[calls] == independent_minimum(calls, (3,) * n), "full observation disagreement") + full = full_costs[calls] + part, cuts = model.restored_partition(calls, masks) + require(part == independent_partition(calls, masks), "partition comparator disagreement") + require(cost <= full <= n and cost <= part <= n, "comparator order") + dirty_checks += check_word(calls, masks, words, cost) + q = eager_boundaries(calls)[-1][:2] + productive = q[0] != 1 and q[1] != 2 + productive_cases += productive + productive_improvements += productive and cost < full + partial_improvements += cost < full + noncommuting_improvements += cost < full and len({c[2] for c in calls}) == 2 + partition_improvements += cost < part + per_length[n] += 1 + hist[(n, cost, full, part)] += 1 + row = [calls, masks, cost, full, part, cuts] + digest.update((json.dumps(row, separators=(",", ":")) + "\n").encode()) + witness_digest.update((json.dumps(words, separators=(",", ":")) + "\n").encode()) + check_controls = controls() + calls, masks = ((0, 0, 0), (1, 0, 1), (1, 0, 0)), (3, 1, 3) + price, words = model.solve(calls, masks, True) + return { + "schema_version": 1, "status": "PASS_PARTIAL_OBSERVATION_BOUNDARIES", + "scope": "exact finite scalar XOR boundary model; no supplier or kappa claim", + "states": 96, "generator_edges": 576, "instances": sum(per_length.values()), + "instances_by_length": {str(k): v for k, v in sorted(per_length.items())}, + "histogram_columns": ["children", "partial_adapter_min", "full_observation_adapter_min", "restored_partition_min", "instances"], + "histogram": [list(key) + [value] for key, value in sorted(hist.items())], + "partial_observation_improvements": partial_improvements, + "noncommuting_partial_improvements": noncommuting_improvements, + "improvements_over_restored_partition": partition_improvements, + "both_outputs_nonzero_cases": productive_cases, + "both_outputs_nonzero_partial_improvements": productive_improvements, + "literal_dirty_input_replays": dirty_checks, + "census_sha256": digest.hexdigest(), "attaining_words_sha256": witness_digest.hexdigest(), + "independent_search": "0-1 BFS over full 4x4 binary maps and chronological layers, all 4808 instances", + "witness": {"children": calls, "masks": masks, "adapter_words": words, + "adapter_cost": price, "fixed_child_xors": 6, "total_xors": 7, + "full_observation_min_total_xors": 9, "restored_partition_min_total_xors": 9}, + "source_inputs_checked": checked_sources, **check_controls, + **short_word_floor(calls, masks), + } + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--write-result", type=Path) + parser.add_argument("--result", type=Path, default=PACKAGE / "RESULT.json") + args = parser.parse_args() + if args.write_result: + require(not args.write_result.exists(), "refusing to overwrite an existing result") + result = certify() + encoded = json.dumps(result, indent=2) + "\n" + if args.write_result: + with args.write_result.open("x", encoding="utf-8") as out: + out.write(encoded) + else: + expected = json.loads(args.result.read_text()) + require(json.loads(encoded) == expected, "receipt mismatch") + print(f"PASS_PARTIAL_OBSERVATION_BOUNDARIES instances={result['instances']} states=96 " + f"partial_gains={result['partial_observation_improvements']} " + f"productive_gains={result['both_outputs_nonzero_partial_improvements']} " + f"dirty_inputs={result['literal_dirty_input_replays']} " + f"controls={len(result['negative_controls'])} no_new_kappa") + + +if __name__ == "__main__": + try: + main() + except (ValueError, OSError, KeyError, TypeError) as exc: + print("FAIL_PARTIAL_OBSERVATION_BOUNDARIES:", exc, file=sys.stderr) + raise SystemExit(1)