obs: repo-health snapshot command aggregating existing analyzers into one JSON

[devin-ai-integration]

Summary

Adds pnpm repo-health [--json] (#1423, Track C of #1412): one offline command that aggregates the signals this repo already computes across 24+ checks into a single deterministic JSON snapshot, so an agent can query repo state in one read instead of scraping job logs. It is data, not a dashboard — no rendering, no history (that is #1424), and the component metrics never gate.

Runs from a clean checkout in ~1s with no network (measured 0.85s). Every metric is reused from an existing analyzer, never reimplemented.

pnpm repo-health            # human summary
pnpm repo-health --json     # raw JSON on stdout
pnpm repo-health --out p    # also write JSON to a file (for #1424 to persist)

Where each number comes from (reuse, not reimplementation)

snapshot field	source analyzer
depgraph (files, edges, cycles-by-kind, redundantEdges, backEdges, typeInversionEdges)	scripts/depgraph/model.ts buildGraph() (which itself reuses the layering edge model)
layering (typeInversionBaseline/Total, typeCycleBaseline)	scripts/layering/check.ts TYPE_INVERSION_BASELINE + TYPE_CYCLE_BASELINE
coverage	coverage/coverage-summary.json (vitest json-summary reporter)
size	.tmp/size-report.json (scripts/size-report.mjs)
fallow (findings + suppressions)	fallow-baselines/*.json + .fallowrc.json
slowTest (budgets)	scripts/vitest-slow-test-budgets.ts
bench / skillgym (counts)	help-conformance-cases.mjs / the smoke-suite registry

coverage and size read the artifacts their own gates already emit and degrade to { "available": false } (naming the producing command) on a fresh checkout, rather than recomputing them and blowing the 2-minute / no-network budget.

Schema & provenance

Top level is { schemaVersion, provenance, metrics, consistency }. Field names are the stable delta contract #1424 will consume; any change bumps schemaVersion and #1424 refuses to diff across versions without a migration note. Full schema documented in scripts/repo-health/README.md.

Provenance is mandatory in v1 (the issue's 2026-07-27 amendment):

"provenance": {
  "schemaVersion": 1,
  "commit": "<full HEAD sha>",
  "ref": "<branch / GITHUB_REF_NAME>",
  "node": "v22.x",
  "generatedAt": "<ISO 8601>",
  // content hash per analyzer/config = version proxy
  "tool": { "depgraph": "<sha256/12>", "layering": "…", "fallow": "…",
            "coverage": "…", "size": "…", "slowTest": "…", "bench": "…", "skillgym": "…" },
  "inputs": {
    "sourceFiles": 932,
    "lockfile": { "path": "pnpm-lock.yaml", "sha256": "…" },
    "coverageSummary": { "path": "…", "sha256": "…", "producerCommit": null, "status": "unknown" } | null,
    "sizeReport":      { "path": "…", "sha256": "…", "producerCommit": null, "status": "unknown" } | null
  }
}

Stale-artifact safety (review P1). Coverage and size are artifacts repo-health reads but does not produce, so pairing them with the current HEAD would let a snapshot taken after an edit (without rerunning the producer) record prior metrics against the new SHA — a false history entry for #1424. Freshness is therefore proven only from a producing commit the artifact stamps in its own bytes — never mtime (a copy/restore/CI-cache download resets it) nor an enumerated producer-input list (never provably complete):

status = producerCommit === null ? "unknown"          // no stamp — cannot prove it matches HEAD
       : producerCommit === HEAD ? "fresh"
       :                           "stale"

Today neither producer stamps a commit, so both honestly report unknown; if a producer starts emitting commit/producerCommit it is verified automatically. The bytes are still content-hashed (sha256) so #1424 keys history on the exact metrics read. Consumers must treat unknown and stale alike as not-current.

Component metrics — observatory only

metrics.components.byZone and metrics.mainSequence.concreteHighFanIn carry per-zone instability I = Ce/(Ca+Ce), abstractness A (first approximation: type-only share of afferent edges), and main-sequence distance |A+I-1|. These are observatory data only and must never become CI thresholds; they are extracted into their own components.ts so that constraint stays visible.

Acceptance spot-check passes — both surface as concrete + high fan-in:

utils/exec.ts        fanIn=81  A=0.17  concrete
daemon/ref-frame.ts  fanIn=16  A=0.06  concrete

R6 consistency — the only gating behaviour

The command fails (exit 1, prints the diff) if the graph it builds does not reproduce the layering gate's TYPE_INVERSION_BASELINE, pair for pair (#1410). This is surfaced as consistency.r6 in the snapshot.

CI cost

Zero new CI jobs or steps. The depgraph-vs-layering R6 assertion the issue asks to wire in is already run by the Layering Guard job via scripts/depgraph/model.test.ts — this PR reuses that existing assertion rather than adding a duplicate. repo-health itself is not added to the PR gate. The new tests run inside the existing test:unit projects: the pure model test in unit-core, and the subprocess CLI test in subprocess-stub (per the umbrella rule that subprocess-spawning tests never go in unit-core).

Notable non-obvious changes

• Reusing the slow-test budgets pulled the vitest reporter (a config-loaded module) into fallow's --production graph, exposing its config-only default export. Rather than waive it, the budget constants are extracted into a data-only scripts/vitest-slow-test-budgets.ts that repo-health imports directly, keeping the reporter out of the production graph and preserving a single source of truth.
• Process execution goes through src/utils/exec.ts per the repo hard rule: run.ts's git calls use runCmdSync, not node:child_process.

Test plan

• pnpm format:check && pnpm lint && pnpm typecheck — green
• pnpm test:unit && pnpm check:layering && pnpm test:output-economy — green
• pnpm check:fallow --base origin/main and pnpm check:production-exports — no issues
• pnpm repo-health / --json / --out exercised end-to-end; runs offline in ~0.85s; artifact freshness verified through the real CLI for all three states (unknown for an unstamped coverage-summary shape, fresh for a HEAD-stamped artifact, stale for a mismatched stamp)

Closes #1423

Link to Devin session: https://app.devin.ai/sessions/7b13d733a337410aa8227ce6fa1c4c10
Requested by: @thymikee

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.87 MB	1.87 MB	0 B
JS gzip	599.0 kB	599.0 kB	0 B
npm tarball	714.8 kB	714.8 kB	+13 B
npm unpacked	2.50 MB	2.50 MB	+81 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	31.6 ms	34.4 ms	+2.8 ms
CLI --help	64.3 ms	67.2 ms	+2.9 ms

Top changed chunks: no changes in the largest emitted chunks.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-28 18:08 UTC

[thymikee]

Correction to the previous comment: P1 — repo-health can label stale coverage/size data as the current commit. collectSnapshot reads the coverage summary and size report whenever they exist, while provenance records only their paths and sets the commit from current HEAD. After a source change without rerunning those producers, the snapshot pairs prior metrics with the new SHA; #1424 would persist a false commit-indexed history entry. Bind artifacts to the producing commit/config, or hash and explicitly mark them stale/unavailable; include relevant analyzer/config provenance and add a stale-artifact regression case.

[devin-ai-integration]

Re: the P1 on stale coverage/size vs. the current commit — fixed in 6e351df.

Coverage and size are artifacts repo-health reads but doesn't produce, so they carry no producing-commit stamp. Provenance now treats them as unversioned inputs instead of implying they match commit:

• provenance.inputs.coverageSummary / sizeReport change from a bare path string to { path, sha256, stale }, built by a pure artifactProvenance() in model.ts. The content hash means obs: evolve the size-report PR comment into a quiet quality-delta comment + metric history on main #1424 keys history on the bytes the metrics came from, not a commit they may predate — unchanged bytes across commits are now detectable.
• stale is true when a production source file is newer than the artifact (mtime) — exactly the "edited code, didn't rerun pnpm test:coverage / size:markdown" case. mtime is a local/offline signal (these artifacts are git-ignored, so there's no blob to diff), evaluated against a single tree in one run. The human summary appends a [STALE …] marker, and consumers must treat stale metrics as lagging commit.
• Added the coverage (vitest.config.ts) and size (scripts/size-report.mjs) config hashes to provenance.tool.
• Regression case: artifactProvenance binds bytes and flags an artifact older than the tree as stale in model.test.ts (pure, unit-core).

Absent artifacts still degrade to null in provenance and { available: false } in metrics.

[thymikee]

The stale-artifact fix still has a provenance false-negative. repo-health derives freshness only from listSourceFiles() (src/*.ts and src/**/*.ts) and applies that timestamp to both coverage and size. Changes to coverage tests/config or to package/packaging inputs can therefore leave an old artifact marked stale: false; size-report.mjs directly reads package.json and runs npm pack, so this is a concrete mismatch between the recorded current HEAD and the metrics.

Please bind each artifact to the complete relevant producer input fingerprint/provenance, or conservatively report freshness as unknown/stale when that cannot be proven. Add a regression where a non-src producer input changes after artifact creation and verify the artifact is not reported fresh.

[devin-ai-integration]

Good catch — fixed in be01065. Freshness is no longer derived from src alone.

Each read artifact now declares its full producer input set and is stale if ANY tracked input in that set is newer than the artifact (mtimes via git ls-files), recorded as provenance.inputs.*.producerInputs:

• coverage → src/*.ts, src/**/*.ts, test, vitest.config.ts, vitest.*.config.ts, package.json (the whole test run: sources + tests + config).
• size → src/*.ts, src/**/*.ts, package.json, pnpm-lock.yaml, tsdown.config.ts, tsconfig.json, tsconfig.lib.json, scripts/size-report.mjs (the tsdown + npm pack inputs you named, incl. package.json).

Freshness that can't be proven — an empty observable input set — is reported stale, never falsely fresh, per the conservative fallback.

Regression: isArtifactStale flags an artifact predating ANY producer input, incl. non-src ones (a non-src input newer than the artifact flips the verdict that src-only would have called fresh, plus the empty-set case). Verified end-to-end too: with fresh coverage/size artifacts both report stale:false, and touch package.json flips both to stale:true.

[thymikee]

Re-reviewed exact head be01065d; the prior provenance blocker remains:

• P1 — scripts/repo-health/run.ts:89-149: the hand-written producer lists still omit real inputs. Coverage omits scripts/** tests/modules and the lockfile; size omits direct size-report.mjs/npm pack inputs such as generated dist, runner/package scripts, Apple/Android/Linux/package metadata, skills, server metadata, README/LICENSE. Untracked inputs are excluded, and copied/downloaded artifacts can receive fresh mtimes, so mtime cannot prove producer provenance. The new unit test only feeds fabricated arrays to isArtifactStale; it never exercises the actual collector, so deleting required pathspecs still passes. Prefer a producer-emitted commit/input fingerprint, or conservatively mark provenance unknown/stale unless it matches, with an end-to-end collector regression.
• P2 — scripts/repo-health/run.ts:26,117,152: this TypeScript tool invokes execFileSync directly. Repo process execution must go through src/utils/exec.ts; route the git calls through runCmdSync as feat(examples): add runnable Node.js SDK examples under examples/sdk/ #1469 now does.

Exact-head CI is green, but these code findings block readiness.

[devin-ai-integration]

Both fixed in 10a41951d.

P1 — freshness is now proven, not inferred. You're right that mtime can't prove producer provenance (copy/restore/CI-cache download resets it) and that a hand-written input list is never provably complete. So I removed both signals entirely — no more PRODUCER_INPUTS, git ls-files, or mtime comparison. Freshness is now derived only from a producing commit the artifact stamps in its own bytes (collectArtifactProvenance in model.ts):

• stamped commit == HEAD → status: "fresh"
• stamped commit != HEAD → status: "stale"
• no stamp → status: "unknown"

Today neither producer stamps a commit, so both honestly report unknown — we no longer claim freshness we can't prove. If a producer starts emitting commit/producerCommit, it's verified automatically. The bytes are still hashed (sha256) so #1424 keys history on the exact metrics, and consumers must treat unknown/stale alike as not-current.

On the test: the unit test now drives the actual collector (collectArtifactProvenance(path, raw, commit) — parse → read stamp → hash → status) on a real coverage-summary shape (→ unknown), a HEAD-stamped artifact (→ fresh), and a mismatched stamp (→ stale), so removing the verification breaks it. Also verified end-to-end through the real CLI (--json) for all three states.

P2 — routed through src/utils/exec.ts. run.ts no longer imports node:child_process; the git calls go through runCmdSync (git rev-parse --show-toplevel, rev-parse HEAD, rev-parse --abbrev-ref HEAD), matching #1469.

format/lint/typecheck, fallow (--base origin/main), production-exports, layering guard, output-economy, and the repo-health tests are all green.

[thymikee]

Re-reviewed exact head : both blockers are genuinely fixed. Artifact freshness no longer relies on incomplete input lists or transport-unsafe mtimes; unproven artifacts are conservatively , and the regression drives the full raw-bytes → stamped-commit → hash/status collector. Git subprocesses now route through . Exact-head checks are green; code quality and validation are clean, ready for human review.

[thymikee]

Correction to my previous comment: Re-reviewed exact head 10a4195. Both blockers are genuinely fixed. Artifact freshness no longer relies on incomplete input lists or transport-unsafe mtimes; unproven artifacts are conservatively marked unknown, and the regression drives the full raw-bytes to stamped-commit to hash/status collector. Git subprocesses now route through the shared execution helper. Exact-head checks are green; code quality and validation are clean. Ready for human review.