feat(proof): harvest bounded per-trial Harbor logs

[echobt]

Summary

P0 Mathis / Architecte SN100: Harbor results.trials[] now carries real per-trial log bodies, not only job-level logs.harbor_run_log / logs.harbor_run_tail.

Guest summarize.collect_trials / build_results harvest Owner-locked metal retain files:

Field	Source (omit if missing/empty; never invent)
agent_log	trial.log, else agent/trajectory.json, else terminus_2.pane (agent/terminus_2.pane alias); agent/stdout.txt only if primaries are missing
verifier_log	verifier/test-stdout.txt
log_sources	relative paths actually read
logs.harbor_run_log / logs.harbor_run_tail	unchanged

Encoded-size budget guard (Greptile P1): acceptance is len(dumps_results(…).encode("utf-8")) of the document we write — not raw field bytes. Invalid UTF-8 replacement chars and JSON escapes can inflate an 8 KiB tail to ~986 KiB. Steps: 8 → 4 → 2 KiB → omit bodies, re-measure after each step. Score still succeeds. harbor_run_tail is not stripped.

MAX_RESULTS_BYTES is 256 KiB for every contract, including generic-custom-v1 (the 512 KiB figure is only the adaptor-source emit probe, not a results allowance).

Log fields stay optional in validate_harbor (string if present).

Tip vs rebake

Guest rebake + Architecte RE-LOCK required. Emit is in-guest summarize.py. Tipping gateway/challenge/ctx/orch alone does not update /opt/proof/runners.

How to verify

python3 deploy/guest/runners/rlm_fc_in_guest_harbor/tests/test_summarize.py
./deploy/scripts/assert-harbor-runner-results-emit.sh
cargo test -p proof-results

Includes test_invalid_utf8_and_escapes_use_encoded_json_size (0xFF / high-escape inflation still fits or omits).

Greptile

• Greptile has reviewed this PR; findings are fixed or answered
• If the bot was silent, I commented @greptileai review

Test plan

• Harbor summarize unit tests (10 / 16 / 33 trial budgets + invalid-UTF-8)
• cargo test -p proof-results
• Full workspace gates

Risk

Guest-side emit until rebake + RE-LOCK. A pack that still overflows after omit (envelope-only) remains fail-closed TooLarge.

Naming

I did not rename BASE_* env vars, deployed paths, GHCR names, or base-*-v1 domain tags.

  

[cursor]

@greptileai review

Tip vs rebake (Subnet Owner): this harvest is guest-side (rlm_fc_in_guest_harbor/harness/summarize.py). After merge, live per-trial agent_log / verifier_log requires:

1. Re-bake the guest image with --runner rlm_fc_in_guest_harbor=deploy/guest/runners/rlm_fc_in_guest_harbor (+ operator overlay / chroot-hook).
2. Architecte RE-LOCK: stage the new ext4, re-pin PROOF_RLM_VM_IMAGE_DIGEST (do not invent a digest).
3. Restart KVM-host agent + proof-challenge.
4. Tip CP so proof-results allows the optional fields (old docs without logs still validate).

Tipping gateway/challenge/ctx/orch alone does not update /opt/proof/runners in the Firecracker guest. Host harvest already copies guest results.json; no host re-walk of trial dirs is needed once the guest emits the bodies.

[cursor]

@greptileai review

Architecture follow-up (locked names unchanged):

• Emit path is guest summarize.collect_trials / build_results.
• MAX_RESULTS_BYTES stays 256 KiB (reverted the 512 KiB bump). 8 KiB × 2 × 33 trials would overflow, so agent_log / verifier_log are last 2 KiB each (132 KiB + 8 KiB harbor_run_tail + envelope).
• Log fields remain optional in validate_harbor (not required). Wrong JSON type if present is still fail-closed.
• Sources / field names unchanged: agent_log ← trial.log, agent/trajectory.json, terminus_2.pane; verifier_log ← verifier/test-stdout.txt.

Guest rebake + Arch RE-LOCK still required after merge; tip CP alone does not update /opt/proof/runners.

[greptile-apps]

Greptile Summary

The Harbor results changes preserve contract-specific size validation, fit optional per-trial logs within the encoded document budget, and redact decoded log content before selecting output tails.

Confidence Score: 5/5

Safe to merge.

The previously reported generic-contract size-limit issue was manually resolved by echobt without explanation. The previously reported oversized Harbor results issue was manually resolved by echobt without explanation. The previously reported Harbor log secret-tail disclosure issue was manually resolved by echobt without explanation.

Files Needing Attention: None.

_{Reviews (8): Last reviewed commit: "fix(proof): redact decoded harbor logs t..." | Re-trigger Greptile}

[cursor]

@greptileai review

Dig follow-up (locked names unchanged: agent_log + verifier_log):

1. 256 KiB budget guard — harvest prefers 8 KiB/field, shrinks to 4 KiB, then omits bodies so a 33-trial pack still scores. Never 503 TooLarge because trial logs overflow. logs.harbor_run_tail is not stripped.
2. Guest image — emit is summarize.py; live only after rebake + RE-LOCK. Crate tip alone is not enough.
3. Sources: trial.log → agent/trajectory.json → terminus_2.pane; agent/stdout.txt only if those primaries are missing. Verifier: verifier/test-stdout.txt.
4. Log fields stay optional in validate_harbor (string if present).

[cursor]

@greptileai review

P1 Keep Results Within Limit — fixed:

• dumps_results is the single serialize path for size check and the file we write.
• fit_results_under_cap re-measures len(json.dumps(…).encode("utf-8")) after each step: 8 → 4 → 2 KiB → omit agent_log / verifier_log.
• Regression: test_invalid_utf8_and_escapes_use_encoded_json_size (10× 0xFF / \" / \x01 tails inflate past 256 KiB before the guard; after main() the file is ≤ 256 KiB and still scores).

P2: MAX_RESULTS_BYTES stays 256 KiB for every contract (generic-custom-v1 included). The 512 KiB figure is MAX_EMIT_PROBE_BYTES (adaptor source scan only), not a Harbor-only results allowance.

Field names / Owner sources unchanged. Guest rebake still required.

[cursor]

@greptileai review

P2 Scope Harbor Size Limit — fixed:

• generic-custom-v1 (and unknown/missing contract) stays at 256 KiB (MAX_RESULTS_BYTES).
• tbench-harbor-v1 / harbor-trials-v1 get 512 KiB (MAX_HARBOR_RESULTS_BYTES) only after contract is identified.
• load_file may read up to the Harbor ceiling so a 257–512 KiB Harbor file can parse; generic over 256 is still TooLarge.
• validate / require_evaluate re-check encoded JSON size against the selected family cap.

Guest summarize still writes ≤ 256 KiB. Field names / Owner sources unchanged.

[cursor]

@greptileai review

P1 Secret Tail Leak — fixed:

• read_tail reads an overlap window (last N plus longest secret + 3 UTF-8 bytes), redacts that buffer, then takes the last N chars. Never redacts only the already-truncated tail.
• Shared by trial agent_log / verifier_log and job-level harbor_run_tail.
• Regression: secret that straddles the cut (naive last-N keeps only a suffix) must not appear in harvested fields.

Field names, encoded-size fit_results, Harbor-only 512 KiB cap unchanged. Guest rebake still required for live emit.

[cursor]

@greptileai review

P1 Secret Tail Leak (UTF-8 remainder) — fixed on 9dd2d13e:

• read_tail no longer sizes the read as max_chars bytes + secret bytes.
• Decode up to 1 MiB from EOF (errors=replace), redact that decoded text, then take the last N characters.
• Same helper for agent_log / verifier_log / harbor_run_tail.
• Regression: secret whose UTF-8 is split on a continuation byte in a last-N-byte window (sk-clé-秘密-owner) must not appear in emitted fields.

Stay draft until this P1 is cleared.

[echobt]

@greptileai

Threads resolved after Safe to merge 5/5 on 9dd2d13. Please re-run status check so merge can proceed.