Skip to content

test(semantics): derive the output-evidence guard from the registry - #7640

Open
xiaoan17 wants to merge 1 commit into
loopx-project:mainfrom
xiaoan17:codex/fix-vocabulary-output-evidence-fixture
Open

xiaoan17 wants to merge 1 commit into
loopx-project:mainfrom
xiaoan17:codex/fix-vocabulary-output-evidence-fixture

Conversation

@xiaoan17

@xiaoan17 xiaoan17 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Scope

One stale test fixture in
tests/architecture/test_semantic_vocabulary_drift.py, and the main-branch
breakage it was hiding.

test_explicit_output_evidence_cannot_be_removed_or_redirected proves that a
vocabulary's explicit output evidence cannot be removed or redirected. It
listed the vocabularies to check by hand:

@pytest.mark.parametrize('metadata,name', [
    ('call_producers', 'loop_disposition'),
    ...
])

PR #7523 retired the Python output builder behind loop_disposition and
dropped that vocabulary's call_producers entry from both the registry and
the drift smoke's CALL_PRODUCER_ANCHOR, but this case kept naming
loop_disposition. With the anchor and the registry both empty for that
vocabulary, the mutation the case performs is a no-op, the coverage floor
raises no drift, and the case fails on its own precondition:

Failed: DID NOT RAISE Drift

This is red on main (2a2323756), so it fails the pytest lane of every open
pull request, including this repository's own.

Fix

The case now reads the anchored vocabularies from the registry instead of
writing them out, and proves the invariant for every vocabulary that
currently carries that metadata:

@pytest.mark.parametrize('metadata', ['call_producers', 'return_paths'])
def test_explicit_output_evidence_cannot_be_removed_or_redirected(metadata):

Two things follow from that.

  • A vocabulary whose evidence is retired with its owner can no longer leave the
    case silently vacuous, which is exactly how this fixture went stale.
  • effective_action carries return_paths and was never in the written-out
    list. It is now covered too, so the case is stronger than the list it
    replaces.

No production code, schema, wire or default change.

Validation

On origin/main 2a2323756:

  • tests/architecture/test_semantic_vocabulary_drift.py — 124 passed
  • ruff check tests/architecture/test_semantic_vocabulary_drift.py — clean

Signed-off-by: xiaoan17 anbc1224@gmail.com

Rebase and re-validation (2026-10-29)

  • Base: main at 9dcdc0bacd455a8cf6f6146f052ffc4df0f5d04a
  • Head: cb7afb2d3a478ab281f3888bffabbafeadc56598
  • The change itself is unchanged; only the parent moved.

This is one of six pull requests that together restore main to green. On main at 9dcdc0bacd455a8cf6f6146f052ffc4df0f5d04a the repository is red in exactly eight places, and this one owns the vocabulary-drift red:

  • tests/architecture/test_semantic_vocabulary_drift.py::test_explicit_output_evidence_cannot_be_removed_or_redirected[call_producers-loop_disposition]

The other seven are owned by #7621, #7631, #7632, #7642 and #7646. None of the eight is caused by a pull request in this set.

All nine pull requests in this set were rebased onto main at 9dcdc0bacd455a8cf6f6146f052ffc4df0f5d04a and merged together in a local worktree. The merged state passes tests/architecture/, tests/canary/, tests/test_manager_context_roundtrip.py, tests/test_lark_native_reply_context.py, tests/control_plane/test_host_adapter_settlement_step_vocabulary.py and tests/control_plane/test_host_adapter_settlement_partial_steps.py — 1805 passed, 0 failed. python scripts/generate_rfc_status_index.py --check is clean.

@xiaoan17
xiaoan17 force-pushed the codex/fix-vocabulary-output-evidence-fixture branch 3 times, most recently from b345385 to 094df1e Compare October 29, 2026 09:11
tests/architecture/test_semantic_vocabulary_drift.py::test_explicit_output_evidence_cannot_be_removed_or_redirected
listed the vocabularies whose explicit output evidence must not be removed
or redirected. PR loopx-project#7523 retired the Python output builder behind
loop_disposition and dropped that vocabulary's call_producers entry from
both the registry and the drift smoke's anchor, but the case here kept
naming loop_disposition.

With the anchor and the registry both empty for that vocabulary, the
mutation the case performs was a no-op, so the coverage floor raised no
drift and the case failed on its own precondition:

```
Failed: DID NOT RAISE Drift
```

The case now reads the anchored vocabularies from the registry instead of
writing them out, and proves the invariant for every vocabulary that
currently carries that metadata. Two things follow. The retired
vocabulary can no longer leave the case vacuous, and effective_action -
which carries return_paths and was not in the written-out list - is now
covered as well.

No production code, schema, wire or default change.

Validation: tests/architecture/test_semantic_vocabulary_drift.py 124
passed on origin/main 0a9a3a3; ruff clean.

Signed-off-by: xiaoan17 <anbc1224@gmail.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

未发现阻塞 finding,建议 APPROVE。对象 cb7afb2d3a478ab281f3888bffabbafeadc56598,基线 9dcdc0bacd455a8cf6f6146f052ffc4df0f5d04a;未查询 GitHub CI。

动机 / Motivation

维护语义注册表的开发者在合法退役一个旧输出 builder 后,会遇到仍要求删除该空声明时必须报错的过期测试。loop_disposition 的 call_producers 已随旧 owner 退役而移除,旧用例把空值再改成空值,无法触发它期待的错误;新用例只枚举当前实际声明输出证据的词表,逐个验证移除会被既有代码锚点拒绝。本次实测恢复了该测试,并补到旧名单遗漏的 effective_action.return_paths。只修已有验证夹具,不改变注册表、扫描器、运行时语义或任何预算。

改动思路 / Approach

没有改产品来迎合旧测试,也没有删除输出证据保护。测试从实际注册表选出非空的两类 metadata,再为每个词表建立独立副本,清空它的输出声明并期待既有 guard 拒绝。拒绝依据仍是代码中单独维护的锚点,不是拿注册表与自己作相等比较。整个类别没有 subject 时也会失败,避免空循环被算成覆盖。

具体改动 / Changes

规范依据是 docs/architecture/rfcs/semantic-vocabulary-convergence-v0.md,spec_revision 9dcdc0bacd455a8cf6f6146f052ffc4df0f5d04a。I8 对应覆盖不能被静默缩小,§5 说明输出参数/路径声明必须与代码锚点一起维护;I10 要求这套 smoke 留在默认 pytest 路径。当前实现满足这两个相关条件,不扩大到任意动态语义的证明。

关键代码讲解

test_explicit_output_evidence_cannot_be_removed_or_redirected 将旧的5组硬编码参数改成两个 metadata 类别,并逐个验证所有当前非空声明。实查 call_producers 有 agent_scope_frontier_action、turn_result_kind 两个 subject;return_paths 有 effective_action、turn_route、turn_result_kind 三个 subject。

check_coverage_floor 未改:逐项要求注册表 metadata 等于 CALL_PRODUCER_ANCHOR / RETURN_PATH_ANCHOR,默认空值也必须匹配。每次 mutation 重新 deep-copy,前一轮失败状态不会污染后一轮。loop_disposition 的旧 builder 已随 #7523 退役,保留那个空值到空值的“删除”测试没有保护作用。

对主干的风险 / Risks

重点反例是自动选 subject 后测试会不会自证。我对5个实际 subject 分别移除与重定向,10组均被原 guard 拒绝;再只在内存中放过 effective_action.return_paths 的错误,新 return_paths 用例确实失败,另一类别仍通过。这验证了旧名单遗漏的 subject 现在能使测试转红。

基线旧参数用例实际为1 failed/4 passed,失败精确对应 loop_disposition 的无效 mutation;当前整个模块124 passed。PASS:标准 premerge 的5项直接检查、2项 catalog 检查及 Ruff。测试数量从5组参数变成2组循环不能当作覆盖缩水或覆盖增加的证据,实际 subject 和反例才是依据。

我的整体评价 / Verdict

long_horizon 与 user_experience 均 preserved:产品语义、原始注册表和预算均不变,只修已有检验的适用对象。仅修改 pytest 验证夹具,没有渲染或产品交互变化;视觉检查不适用。静态输出锚点检查不能证明任意动态 Python/TypeScript 语义或整体发布资格。同作者其它修复有各自 owner 和验证边界,本案没有重复造 smoke。剩余限制是代码锚点与扫描器仍有其声明范围,124项通过不等于全项目发布已合格。

English verdict: APPROVE cb7afb2;124 checks pass, all10 live metadata mutations are rejected, and a selective effective_action guard bypass is detected by the new test.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants