diff --git a/CHANGELOG.md b/CHANGELOG.md index 7e5b83c..b65ba1d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,17 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and ### Added +- M4 ENV-024–ENV-026: scrolling Bubble Tea reports with plain fallback, explicit + `--format text|markdown|json`, `--plain` and escaped, bounded Markdown export. +- Version-1 diagnostic bundles with typed allowlisted sharing evidence, opaque path/host aliases, + Git identity presence facts, explicit omissions, strict archive/JSON bounds and atomic publication. + `--include-logs` contains only structured collector statuses/timings. +- Offline `diagnose --from-bundle`, recomputing current builtin findings from validated evidence with + capture/evaluator provenance and no collectors or network work. +- Accepted [ADR 0006](docs/adr/0006-m4-reports-and-bundles.md), sharing schemas, archive/privacy/renderer/ + CLI tests, a real terminal capture and [M4 validation](docs/testing/m4-validation.md). + Local gates and native M4 Windows/Ubuntu/macOS GitHub CI passed; + [PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) is open for review. - M3 rules ENV007–ENV012: known Docker API incompatibility, explicitly required dependency absence, package-manager/lockfile disagreements, effective Git identity/explicit autocrlf policy, canonical virtualenv/interpreter mismatch and opt-in proxy refusal/no-route evidence. @@ -21,7 +32,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and CLI, cancellation and socket lifecycle regressions. - Accepted [ADR 0005](docs/adr/0005-m3-evidence-and-rule-registry.md), [M3 acceptance requirements](docs/phases/M3.md), [validation evidence](docs/testing/m3-validation.md) - and [development-tool audit](docs/DEVELOPMENT-TOOLS.md). M3 is complete on the feature branch; + and [development-tool audit](docs/DEVELOPMENT-TOOLS.md). M3 is complete and merged through PR #3; [CI 43723839978](https://github.com/Shellghostt/EnvDoctor/actions/runs/43723839978) passed native Windows/Ubuntu/macOS race, tests and builds, plus lint and the vulnerability scan. @@ -35,6 +46,9 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and ### Changed +- Optional bundle-source metadata extends report schema v1 and stays absent from ordinary reports. +- Terminal dependencies are pinned/audited, including Bubble Tea v1.3.10, x/term v0.48.0, + x/sys v0.50.0 and x/text v0.44.0; see [M4 dependency review](docs/testing/m4-dependencies.md). - Optional M3 snapshot evidence extends schema v1 compatibly; project and rule-registry JSON have separate version-1 schemas. Explicit typed partial collector evidence retains degraded notices and disables dependent rules. @@ -72,7 +86,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and ### Planned -- M4 reports and bundles; later rules ENV013–ENV018, confirmed fixes and signed releases. +- Rules ENV013–ENV018, confirmed fixes and signed releases. ## [0.1.0] - 2026-11-13 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d139b3e..5b4daa1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,8 +9,8 @@ Thanks for taking the time. This document is deliberately short and specific. 2. [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) - how the pieces fit together. 3. [docs/adr/](docs/adr/) - decisions that have already been argued about. Do not relitigate them without a new ADR. -4. [docs/phases/M3.md](docs/phases/M3.md) and [ADR 0005](docs/adr/0005-m3-evidence-and-rule-registry.md) - - implemented M3 evidence contracts and verified three-OS CI gate. +4. [docs/phases/M4.md](docs/phases/M4.md) - report/bundle contracts, dependency order and acceptance gates; + preserve [M3 evidence contracts](docs/phases/M3.md) and accepted ADRs 0005/0006. 5. [docs/DEVELOPMENT-TOOLS.md](docs/DEVELOPMENT-TOOLS.md) - CodeGraph freshness, static-analysis coverage and profiling appropriate to this repository's runtime. @@ -36,6 +36,8 @@ the CLI and runs all 172 golden cases through it, including the unchanged 54 M2 Each new rule needs positive, negative and edge snapshots for Windows, Linux and macOS. Expected findings are reviewed evidence, not output regenerated from the implementation. See [M3 validation](docs/testing/m3-validation.md) for native Windows and Linux race commands and limits. +[M4 validation](docs/testing/m4-validation.md) records terminal, sharing, archive, offline and resource +cases. M4 local gates and native Windows/Ubuntu/macOS GitHub CI pass. Windows race tests need a 64-bit MinGW-w64-compatible GCC configured in Go's CC/CXX; ordinary static builds use `make build`, which disables CGO only for that target. diff --git a/README.md b/README.md index 9af01b7..eabc1d2 100644 --- a/README.md +++ b/README.md @@ -2,16 +2,21 @@ > A cross-platform CLI that detects, diagnoses and helps resolve development environment problems. -![status](https://img.shields.io/badge/status-M3_complete-green) +![status](https://img.shields.io/badge/status-M4_verified-green) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE) ![Go](https://img.shields.io/badge/go-1.26-00ADD8?logo=go&logoColor=white) -**Status: M3 complete on `feat/m3-implementation`; Windows, Ubuntu and macOS CI passed.** +**Status: M1–M3 merged; M4 implemented and verified on `feat/m4-implementation`.** +Native Windows/Ubuntu/macOS race, tests and builds, lint and the vulnerability scan passed in +[M4 CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/44421272870). +[PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) is open for review. Nine collectors feed twelve rules covering platform, shell, PATH, Python/Node runtime, requested TCP ports, effective Git config, local Docker/API metadata, explicit project dependencies and proxy configuration. -Reports are plain text or versioned JSON; `explain` and `rules --json` work offline. +Reports support a scrolling terminal viewer, plain text, Markdown and versioned JSON. +Redacted diagnostic bundles can be evaluated offline; `explain` and `rules --json` also work offline. Diagnosis never applies fixes or executes interpreters found on PATH. The -[M3 validation record](docs/testing/m3-validation.md) records commands, live checks and limits. +[M4 validation record](docs/testing/m4-validation.md) records commands, live checks and limits; +[M3 validation](docs/testing/m3-validation.md) preserves the merged baseline evidence. ## Why this exists @@ -35,8 +40,8 @@ explains the cause instead of dumping raw output. | Detect virtualenv/interpreter mismatch | M3 | **Implemented**, known canonical environment metadata | | Inspect proxy configuration and optional reachability | M3 | **Implemented**, redacted and offline by default | | Explain rules and export the rule registry | M3 | **Implemented**, no collectors invoked | -| Generate diagnostic bundles | M4 | Not started | -| Human-readable reports | M2/M4 | **Done** plain text; interactive polish is M4 | +| Generate redacted diagnostic bundles and evaluate them offline | M4 | **Implemented and verified** locally and in native CI | +| Human-readable reports | M2/M4 | Plain text, scrolling terminal viewer and escaped Markdown | | JSON export | M1 | **Done** | | Safe, opt-in automated remediation | M5 | Not started | | Windows, Linux and macOS support | M2/M6 | Three-OS code and fixtures; signed release/clean-host verification is M6 | @@ -49,7 +54,10 @@ flowchart TD COL --> SNAP[Typed snapshot and redaction] SNAP --> ENGINE[Pure diagnostic engine] REG[Rule registry] --> ENGINE - ENGINE --> REPORT[Plain text or JSON with manual advice] + ENGINE --> REPORT[Terminal, text, Markdown or JSON report] + SNAP --> BUNDLE[Allowlisted sharing evidence and bounded ZIP] + BUNDLE --> OFFLINE[Validated captured evidence] + OFFLINE --> ENGINE META[explain and rules] --> REG REG --> PAGES[Generated rule pages] ``` @@ -110,6 +118,51 @@ in the test suite; the snapshot inside it is described by [docs/schema/snapshot-v1.schema.json](docs/schema/snapshot-v1.schema.json). Home directories are replaced with `~`, and credential-bearing URLs are redacted before anything is written. +## Terminal reports and sharing bundles + +The default report opens a scrolling viewer when stdin, stdout and stderr are terminals. Use arrows, +Page Up/Page Down and `q`; quitting retains the diagnosis exit code and leaves a plain report behind. +Interrupts and rendering failures exit 3 and restore terminal state. Redirects, `TERM=dumb`, `NO_COLOR` +(including an empty value), `--plain` and explicit text select plain output. + +![EnvDoctor M4 terminal viewer](docs/images/m4-terminal.jpg) + +This is real Windows PTY output from the implemented CLI, displayed in xterm.js with the committed +synthetic [viewer fixture](docs/testing/fixtures/m4-viewer/snapshot.json). Its identities and paths are +examples. The browser display is a captured terminal preview; native console restoration and resize +were checked separately. + +```powershell +.\bin\envdoctor.exe diagnose --plain +.\bin\envdoctor.exe diagnose --format markdown > diagnosis.md +.\bin\envdoctor.exe diagnose --format json > diagnosis.json +.\bin\envdoctor.exe bundle --output envdoctor-bundle.zip --include-logs +.\bin\envdoctor.exe diagnose --from-bundle envdoctor-bundle.zip --json +``` + +JSON and Markdown bypass the viewer. `--json` remains an alias for `--format json`; conflicting formats +or `--plain` with JSON/Markdown exit 4 before collection. Markdown includes evidence, manual suggestions, +verification, limitations, degraded collectors and skipped prerequisites, with dynamic markup escaped. + +Bundle export uses a separate typed allowlist: Git identity becomes presence booleans, paths and hosts +become deterministic capture-local aliases, and unsupported required evidence is explicitly omitted and +degraded. Raw environment values, identities, project contents and free-text logs/errors are excluded. +`--include-logs` adds only structured collector statuses and bounded timings. Numeric versions, ports and +diagnostic facts still describe the captured setup; review the bundle before sharing it. + +The ZIP is below 2,000,000 bytes, with independent member, decompression and JSON-work bounds. Publication +stages a private file and replaces the destination atomically after validation. The bounded `Writing...` +status precedes replacement; exit 0 confirms publication, including captures with findings or degradation. +Usage exits 4; capture, archive, cancellation, output and file failures exit 3. + +`--from-bundle` validates evidence and evaluates the current builtin rules without collectors or network +work. It preserves capture provenance and omissions; saved verdicts and commands remain reference data. +It conflicts with `--fixture-dir`, `--port`, `--probe-network` and `--package-manager`. Valid offline +diagnosis retains exits 0/1/2/3, including 3 for unavailable required evidence. Ordinary JSON keeps its +existing fields and Git identity; bundle-source metadata is an optional addition used by offline reports. +Read [ADR 0006](docs/adr/0006-m4-reports-and-bundles.md) and the +[sharing schemas](docs/schema/bundle-snapshot-v1.schema.json) for the exact policy and limits. + Exit codes: `0` nothing at or above warning, `1` warnings, `2` at least one error, `3` the tool itself failed (including a collector timeout), `4` invalid usage. Captured collector errors retain exit 3 during offline diagnosis. A degraded report still contains the findings that could be evaluated. @@ -188,7 +241,7 @@ Different Docker product versions alone do not establish API incompatibility. Wi opaque shims and protected package paths remain unresolved metadata. Home paths and credential-bearing URLs are redacted, including error notices. JSON includes Git identity; -review it before sharing. Diagnostic bundles with broader redaction remain M4. +review it before sharing. Use the M4 bundle command for the separate allowlisted sharing representation. ## Testing @@ -206,8 +259,10 @@ The suite includes 172 exact golden cases, preserving all 54 original M2 snapsho It builds the CLI and checks those cases through real subprocesses, validating JSON and exit codes. Native Windows and Linux race suites passed locally. The native Windows, Ubuntu and macOS race/test/build matrix, lint and vulnerability scan passed in [M3 CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43723839978). -Both macOS architectures also cross-build. Commands, allocation measurements, -socket lifecycle evidence and limits are in [docs/testing/m3-validation.md](docs/testing/m3-validation.md). +M4 passes local Windows/Linux race, all canonical targets, bounded fuzzing, archive/privacy tests and +all six static build targets. Native Windows/Ubuntu/macOS M4 CI also passed. Commands, terminal evidence, +allocation measurements and limits are in [M4 validation](docs/testing/m4-validation.md); +[M3 validation](docs/testing/m3-validation.md) records the earlier socket and CI evidence. `make docs` regenerates all twelve rule pages from the same registry as findings, `explain` and `rules`. `make check-docs` and CI detect documentation drift. Do not edit generated pages directly. @@ -218,7 +273,8 @@ A local release build of all six targets: goreleaser build --snapshot --clean ``` -`goreleaser check` and the local six-target snapshot build passed. A snapshot build does not publish a release. +M3 validation records the passing GoReleaser check and six-target snapshot build. M4 verifies static +builds separately; packaging, signed releases and clean-host checks remain M6. ## Repository layout @@ -230,9 +286,10 @@ goreleaser build --snapshot --clean | `internal/config` | Resolved options, schema version, collector timeout | | `internal/collector` | Parallel runner and nine collectors, including project and proxy evidence | | `internal/diagnostic` | Rule registry, pure engine, severity model, fixture harness | -| `internal/report` | Terminal renderer, JSON renderer | +| `internal/report` | Terminal lifecycle, plain text, Markdown and JSON renderers | +| `internal/bundle` | Allowlisted export, strict bounded reader and atomic publication | | `internal/fix` | Remediation plans, dry-run by default (empty until M5) | -| `internal/system` | Thin wrappers over OS facilities, the only place syscalls live | +| `internal/system` | Machine-inspection wrappers; bundle file I/O and terminal control have separate boundaries | | `docs/schema` | Committed JSON schemas, enforced by tests | | `tests/golden` | 172 snapshot-to-diagnosis fixtures | | `tests/integration` | Built CLI tests against all golden cases | @@ -241,16 +298,21 @@ goreleaser build --snapshot --clean ## Roadmap -M3 is **complete on the feature branch, including three-OS CI**. Read the +M3 is **complete and merged** through [PR #3](https://github.com/Shellghostt/EnvDoctor/pull/3), with +successful [main CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43802280915). Read the [M3 acceptance gates](docs/phases/M3.md), [accepted ADR 0005](docs/adr/0005-m3-evidence-and-rule-registry.md) and [validation record](docs/testing/m3-validation.md) for its contracts and evidence. +M4 ENV-024–ENV-026 are implemented and verified under the +[M4 requirements](docs/phases/M4.md) and [accepted ADR 0006](docs/adr/0006-m4-reports-and-bundles.md). +[Validation](docs/testing/m4-validation.md) records local evidence and passing native GitHub CI; +[PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) is open for review. | Milestone | Deliverable | Status | |---|---|---| | M1 | CLI foundation, collector interface, system information | **Done** (v0.1.0) | | M2 | Cross-platform collectors and rules ENV001–ENV006 | **Done (merged)** | -| M3 | Rules ENV007–ENV012, Docker/dependency/proxy evidence, offline registry commands | **Done (feature branch, CI passed)** | -| M4 | Polished terminal output, Markdown export, diagnostic bundles | Not started | +| M3 | Rules ENV007–ENV012, Docker/dependency/proxy evidence, offline registry commands | **Done (merged)** | +| M4 | Terminal viewer, Markdown, redacted bundles and offline diagnosis | Verified; [PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) open | | M5 | Safe, confirmed automated remediation | Not started | | M6 | Windows, Linux, macOS packaging and signed releases | Not started | diff --git a/cmd/envdoctor/main.go b/cmd/envdoctor/main.go index 3646498..7c21c78 100644 --- a/cmd/envdoctor/main.go +++ b/cmd/envdoctor/main.go @@ -6,7 +6,10 @@ package main import ( + "context" "os" + "os/signal" + "syscall" "github.com/shellghostt/envdoctor/internal/cli" ) @@ -17,10 +20,15 @@ import ( var version = "dev" func main() { + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() os.Exit(cli.Main(os.Args, cli.App{ - Version: version, - Stdout: os.Stdout, - Stderr: os.Stderr, - Getenv: os.Getenv, + Version: version, + Stdin: os.Stdin, + Stdout: os.Stdout, + Stderr: os.Stderr, + Getenv: os.Getenv, + LookupEnv: os.LookupEnv, + Context: ctx, })) } diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 2a76f08..e979e06 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,8 +1,8 @@ # EnvDoctor Architecture -**Status: M3 complete on the feature branch; three-OS CI passed.** Nine collectors feed -twelve pure rules and a plain-text/JSON report. Offline metadata commands and generated pages share the -same rule registry. Bundles, automatic fixes and interactive output remain planned. +**Status: M1–M3 merged; M4 implemented and verified, including native three-OS GitHub CI.** +Nine collectors feed twelve pure rules and terminal/plain/Markdown/JSON reports. Offline metadata commands, +generated pages and validated bundle diagnosis share the same registry. Automatic fixes remain planned. The authoritative specification is [PLAN.md](PLAN.md); [ADR 0005](adr/0005-m3-evidence-and-rule-registry.md) records the M3 extensions to the M2 evidence and inspection boundaries. @@ -25,7 +25,11 @@ flowchart TD SNAP --> REDACT[Home and credential URL redaction] REDACT --> ENGINE[Pure rule evaluation] REG[Immutable builtin registry] --> ENGINE - ENGINE --> REPORT[Plain text or JSON with manual advice] + ENGINE --> REPORT[Terminal, text, Markdown or JSON with manual advice] + REDACT --> EXPORT[Typed allowlisted sharing model] + EXPORT --> ZIP[Bounded validated ZIP publication] + ZIP --> READ[Strict archive and evidence validation] + READ --> ENGINE FIXTURE[Offline versioned snapshot] --> REDACT META[explain and rules] --> REG GEN[cmd/rulegen] --> REG @@ -76,13 +80,13 @@ The generator's check mode is enforced by tests and CI. Optional M3 snapshot fie project and registry JSON have separate version-1 schemas. Redaction covers home paths and credential-bearing URLs, including errors. Git identity remains part of -the JSON snapshot under the M2 sharing contract. The future bundle feature must define its own sharing -policy. No current command applies a fix. +the JSON snapshot under the M2 sharing contract. Bundles use the separate M4 sharing policy. +No current command applies a fix. ## What is deliberately absent -See [PLAN.md](PLAN.md) section 4 (non-goals). If a component is not on that diagram, it is not part of the -design. +See [PLAN.md](PLAN.md) section 3 for non-goals and section 4 for release scope. Rules ENV013–ENV018, +automatic fixes/undo and signed packaging remain later milestones. ## M3 evidence @@ -91,3 +95,36 @@ Read [phase requirements](phases/M3.md), [ADR 0005](adr/0005-m3-evidence-and-rul and Linux race suites passed locally; native Windows/Ubuntu/macOS tests and race suites also passed in [M3 CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43723839978). [Development tools](DEVELOPMENT-TOOLS.md) records navigation and runtime-profiling coverage. + +## M4 report and bundle boundaries + +[M4 requirements](phases/M4.md) and [accepted ADR 0006](adr/0006-m4-reports-and-bundles.md) define +Bubble Tea/plain/Markdown output and bounded versioned bundles, implemented at `c532dc3`. +The CLI resolves format conflicts before collection. Automatic viewer selection requires all three +streams to be terminals; explicit formats bypass terminal/environment detection. The report adapter +owns terminal state, output-error cancellation and a joined 200 ms resize poller. Every viewer exit +restores state and writes plain output. Markdown escapes dynamic content and bounds its buffer to 2 MiB. + +`internal/bundle` owns selected-file I/O, private temporary staging and same-directory atomic replacement. +These file operations and `internal/report` terminal control are explicit OS boundaries alongside the +existing machine-inspection wrappers. The pure rule engine and sharing-model conversion inspect no host. + +The exporter copies reviewed facts into a separate model, substitutes equality/presence aliases and +records fixed omissions. A source-model coverage guard rejects unreviewed collector fields. Bundle +reports are recomputed from sharing evidence. Raw errors/logs/identity and arbitrary original findings +are excluded; optional diagnostics contain only allowlisted statuses/timings. + +The reader preflights the complete ZIP directory/local records, bounds streamed Store/Deflate input, +checks CRC/digests and cross-member metadata, then validates exact typed JSON tags and semantic constraints. +Tests compile committed schemas with network loading forbidden. Archives are read without extraction. +Only the sanitized snapshot and provenance reach offline diagnosis; the current registry recomputes +findings. Captured OS normalization preserves ENV011 equality without receiving-host lookups. + +Windows publication applies a protected current-user/SYSTEM ACL before writing payload bytes and uses +`MoveFileEx`; Unix uses mode 0600 and rename. Failure paths remove the owned stage and preserve the prior +destination. Atomic namespace replacement assumes a supporting local filesystem; network filesystem +and power-loss guarantees are outside this format. + +[M4 validation](testing/m4-validation.md) records native terminal checks, cross-OS captures, bounds, +privacy, lifecycle measurements and passing GitHub CI. Existing collectors, rule pages, 172 golden +fixtures and ordinary JSON/fixture behavior remain compatible. diff --git a/docs/DEVELOPMENT-TOOLS.md b/docs/DEVELOPMENT-TOOLS.md index e8db8f6..fe25cce 100644 --- a/docs/DEVELOPMENT-TOOLS.md +++ b/docs/DEVELOPMENT-TOOLS.md @@ -1,8 +1,10 @@ # Development tools and memory validation -Initial audit: 2026-11-14; M3 verification refresh: 2026-11-15. +Initial audit: 2026-11-14; M3 verification refresh: 2026-11-15; M4 verification: 2026-11-19. These are development aids, not product features or substitutes for tests. -Read [PLAN.md](PLAN.md), the [M3 requirements](phases/M3.md) and accepted [ADRs](adr/) first. +Read [PLAN.md](PLAN.md), the [M4 requirements](phases/M4.md), implemented [M3 contract](phases/M3.md) +and accepted [ADRs](adr/) first. M4 ADR 0006 is accepted; [M4 validation](testing/m4-validation.md) +records implemented behavior, local evidence and passing native Windows/Ubuntu/macOS M4 GitHub CI. ## CodeGraph @@ -15,6 +17,12 @@ codegraph status codegraph sync ``` +On 2026-11-17, `codegraph status` reported the existing index up to date against merged M3 source. +No new index or generated source was created for M4 preparation. +On 2026-11-19, `codegraph sync` refreshed the existing index from final M4 implementation `c532dc3`; +`codegraph status` reported **100 files, 1,291 nodes and 3,120 edges**, up to date. No index initialization +or tool upgrade was needed. These counts describe navigation coverage, not complete analysis. + For indexed code, use the CodeGraph MCP exploration tool with this repository's absolute `projectPath` and the symbol/flow you need. Read its returned source and callers before edits; use direct source/config searches for areas the graph does not capture. Unresolved references remain, @@ -42,6 +50,30 @@ Profile only when the change or suspected fault justifies it; a documentation ed profile, goroutine counts and local diagnosis timings. Those are bounded Go workload measurements; MemGuard/MAT were not treated as Go runtime gates or evidence of universal leak freedom. +## M4 implementation validation + +[M4 validation](testing/m4-validation.md) records executed synthetic archive/viewer lifecycles, +rejection, cancellation, native terminal resize/exit and independent decompression/report-buffer bounds. +[Dependency review](testing/m4-dependencies.md) records pinned terminal modules and license evidence. +Terminal dependencies remain Go code; JavaScript static analysis does not cover these paths. + +With `GOMAXPROCS=4`, 100 bundle build/read/cancel cycles took **303.2657 ms** with a 7,342-byte archive. +Goroutines stayed **2 → 2** and retained heap after GC changed **1,797,600 → 887,320 bytes**. +Fifty viewer quit/cancel cycles took **533.4757 ms**; goroutines stayed **2 → 2**, with retained heap +**304,424 → 285,560 bytes**. The resize-poller cancellation/join test also passed. Twenty-five hidden +decompression bombs were rejected in **1.0419 ms**, allocating **2,140,424 bytes total**. + +A 100-iteration bundle benchmark measured **3,002,830 ns/op**, **1,535,852 B/op** and **11,419 allocs/op**. +The sampled retained Go profile showed runtime and test-schema initialization allocations. These are +bounded workload observations; they do not establish leak freedom for every input. Profiles and raw +captures stay local beneath ignored `bin/`; the validation record includes reproduction commands. + +Ten read-only Windows diagnoses on `c532dc3` took **236.06–561.36 ms**, all with nine collectors, +twelve rules and diagnosis exit 1. Windows `GetProcessMemoryInfo` reported peak working sets of +**13,213,696–14,782,464 bytes**, including successful final reads after exit. These local warm runs +met the 3 s/100 MB targets; native macOS interactive/performance, cold-host and release measurements +remain separate work. + ## Memory Analyzer The locally installed analyzer is Eclipse MAT 1.17.0, configured with Java 21 and a 4 GiB analyzer heap. @@ -55,4 +87,3 @@ State the branch and commit, read the phase/ADRs, refresh the graph after source the exact failing command, output and minimal synthetic fixture. Keep implementation on a feature branch. A skipped integration test must be reported as skipped, with its missing dependency; it is not a passing check. Preserve local Docker volumes and use disposable databases for migration tests. - diff --git a/docs/PLAN.md b/docs/PLAN.md index 66c4d97..50441cb 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -12,7 +12,7 @@ depends_on: - No runtime dependencies (single static binary) - Docker (optional, only for testing rules that inspect Docker) created: 2026-11-13 -updated: 2026-11-15 +updated: 2026-11-19 --- # EnvDoctor — Implementation Plan @@ -24,19 +24,26 @@ updated: 2026-11-15 ## 0. Machine context -M2 is complete and merged into `main`. M3 is **complete on `feat/m3-implementation`, including -[three-OS CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43723839978)**. +M1–M3 are complete and merged into `main`. [M3 PR #3](https://github.com/Shellghostt/EnvDoctor/pull/3) +merged at `6cf358138ecd3bedf086f22536a30562fb673e4e`; its +[main CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43802280915) passed. [ADR 0005](adr/0005-m3-evidence-and-rule-registry.md) was accepted and committed before implementation. The [M3 requirements](phases/M3.md) define the evidence and acceptance contract; [M3 validation](testing/m3-validation.md) records the final commands and limits. The [development-tool audit](DEVELOPMENT-TOOLS.md) records actual coverage and profiling limits. +M4 is **implemented and verified** on `feat/m4-implementation` at `c532dc3`, including the +published preparation commit `2be420a`. Native M4 GitHub CI passed; +[PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) is open for review. +[M4 requirements](phases/M4.md) and accepted [ADR 0006](adr/0006-m4-reports-and-bundles.md) define +formats, allowlisted bundle export and offline diagnosis. [M4 validation](testing/m4-validation.md) +records executed local gates, bounds, privacy, terminal evidence and passing native GitHub CI. ```yaml project: EnvDoctor status: mvp version: 0.1.0 builds: true -functional: partial # M1-M3 collection and twelve rules work; M4-M6 remain +functional: partial # M1-M4 verified; M5-M6 remain implemented: - cobra CLI with diagnose, version, offline explain and rules --json @@ -55,6 +62,9 @@ implemented: - one builtin rule registry for findings, explain, rules JSON and generated documentation - home-path and credential-URL redaction including embedded collector errors - text report and a versioned JSON document + - explicit text/markdown/json formats, plain fallback and a scrolling Bubble Tea viewer + - bounded allowlisted ZIP bundles with atomic publication and structured optional diagnostics + - diagnose --from-bundle with current-rule evaluation, capture provenance and explicit omissions - documented exit codes 0/1/2/3/4 - compatible snapshot/report schema v1 additions and separate project/registry v1 schemas - 172 golden fixtures including the unchanged 54 M2 cases and 118 additional cases @@ -65,11 +75,9 @@ implemented: not_implemented: - diagnostic rules ENV013-ENV018 - - bubble tea output - - diagnostic bundles - fixes and undo - signed releases and install docs - - --debug-snapshot and --from-bundle + - --debug-snapshot source_of_truth: scope_and_rules: docs/PLAN.md # this file, section 9 for rule ids @@ -191,6 +199,10 @@ flowchart TD ENGINE --> RULES[Rule engine] RULES --> REPORT[Terminal report] RULES --> JSON[JSON export] + RULES --> MARKDOWN[Escaped Markdown report] + ENGINE --> BUNDLE[Allowlisted evidence and bounded ZIP] + BUNDLE --> OFFLINE[Validated captured evidence] + OFFLINE --> ENGINE RULES --> FIX[Fix recommendations] FIX -->|explicit --yes| APPLY[Apply fix] ``` @@ -206,7 +218,8 @@ flowchart TD | Network inspector | Listening ports, proxy variables, DNS reachability | Make outbound requests beyond a documented probe | | Rule engine | Turn a snapshot into findings | Touch the OS | | Explanation composer | Detected / cause / fix / verify text | Speculate beyond the rule's declared evidence | -| Report renderer | Human text, JSON, diagnostic bundle | Add facts not present in findings | +| Report renderer | Terminal/plain text, Markdown and JSON | Add unobserved diagnostic facts | +| Bundle exporter/reader | Reviewed sharing facts, independent bounds and atomic publication | Extract or execute archive members | | Fix engine | Plan, preview, apply, verify a remediation | Run without `--yes` | ### Flow @@ -226,7 +239,7 @@ The builtin registry owns the twelve pure evaluators and their public metadata. ## 7. Data model -There is no database. The two persisted artefacts are the snapshot and the report. +There is no database. Snapshot/report JSON remain schema v1; M4 adds a separate versioned sharing ZIP. **Snapshot** (internal Go struct, JSON-serialisable, redacted before export): @@ -283,7 +296,7 @@ probe TCP port 2375. See ADRs 0004/0005 and the committed schemas for field and ## 8. Command surface and exit codes ``` -envdoctor diagnose [--json] [--component ] [--rule ] [--verbose] [--port ] [--fixture-dir ] [--probe-network] [--package-manager [@]] +envdoctor diagnose [--format ] [--plain] [--json] [--component ] [--rule ] [--verbose] [--port ] [--fixture-dir | --from-bundle ] [--probe-network] [--package-manager [@]] envdoctor explain envdoctor bundle [--output ] [--include-logs] envdoctor fix [--rule ] [--dry-run] [--yes] @@ -291,10 +304,26 @@ envdoctor rules [--json] envdoctor version ``` -Implemented as of M3: `diagnose`, `version`, offline `explain` and `rules`/`rules --json`. -`bundle` and `fix` remain planned; see section 0 for the authoritative list. Unknown rule IDs and invalid +Implemented through M4: `diagnose`, `bundle`, `version`, offline `explain` and `rules`/`rules --json`. +`fix` remains planned; see section 0 for the authoritative list. Unknown rule IDs and invalid metadata arguments exit 4; output failures exit 3. +M4 format/plain/bundle-input flags and bundle command semantics are accepted in +[ADR 0006](adr/0006-m4-reports-and-bundles.md) and implemented at `c532dc3`. JSON/Markdown bypass the +viewer; explicit plain/text and unsuitable terminal streams fall back to plain output. Format conflicts +exit 4 before collection. Viewer quit retains the diagnosis exit; interrupt/render failure exits 3 and +restores terminal state with a plain fallback. + +`bundle` captures the complete builtin report with the existing capture controls. Its separate sharing +policy removes raw identifying/free-text values. `--include-logs` contains only structured collector +statuses/timings. The pre-publication status includes byte count and original diagnosis exit; successful +atomic publication exits 0. Capture/archive/cancellation/output/I/O failures exit 3; usage exits 4. + +`diagnose --from-bundle` runs no collectors, commands, home/path inspection or network work. It evaluates +validated captured evidence under the current registry, retaining capture metadata and explicit omissions. +It rejects fixture/requested-port/probe/package-manager flags and preserves the diagnosis exit mapping. +Ordinary JSON continues to include effective Git identity; its existing fields/values are unchanged. + | Exit code | Meaning | |---|---| | 0 | No findings at or above `warning` | @@ -484,7 +513,7 @@ See [testing/m1-m2-validation.md](testing/m1-m2-validation.md) for commands, mea **Deliverable:** rules ENV007–ENV012, Docker and dependency inspectors, `explain` and `rules --json`. -Status: **Complete on `feat/m3-implementation`, including native Windows/Ubuntu/macOS CI.** +Status: **Complete and merged into main**, including native Windows/Ubuntu/macOS CI. Follow [M3 requirements](phases/M3.md), [ADR 0005](adr/0005-m3-evidence-and-rule-registry.md) and [M3 validation](testing/m3-validation.md) for evidence, tests and limits. @@ -511,13 +540,23 @@ tests and records local measurements without claiming M6 clean-host/release cove **Deliverable:** polished terminal output (Bubble Tea when TTY), Markdown export, `bundle` command. +Status: **Implemented and verified; native M4 GitHub CI passed, PR #4 open for review.** +ADR 0006 was accepted before dependent code. Follow [M4 requirements](phases/M4.md) and +[accepted ADR 0006](adr/0006-m4-reports-and-bundles.md). Bundle input/export has +its own version/redaction/omission policy. [M4 validation](testing/m4-validation.md) maps acceptance to executed evidence. + **Acceptance criteria** -- [ ] Non-TTY output is plain text, stable, and contains no ANSI codes -- [ ] `envdoctor bundle` produces a zip under 2 MB containing report, redacted snapshot and tool versions -- [ ] Bundle never contains environment variable values, tokens or usernames (test asserts this) -- [ ] A bundle from another machine can be diagnosed offline with `envdoctor diagnose --from-bundle` -- [ ] One screenshot in the README, taken from a real run +- [x] Non-TTY output is plain text, stable, and contains no ANSI codes +- [x] TTY/NO_COLOR/redirect/format and Markdown escaping cases preserve reports/exits and terminal cleanup +- [x] `envdoctor bundle` produces a versioned zip below 2,000,000 compressed bytes with independent entry/decompression limits +- [x] Allowlisted export removes synthetic raw environment values, tokens and identities from every member/error path +- [x] Removed evidence is explicitly omitted/inconclusive; saved report verdicts never override validated captured evidence +- [x] Cross-OS `diagnose --from-bundle` runs no collectors, commands or network and preserves capture/evaluator provenance +- [x] Corrupt/traversal/bomb/duplicate/version/oversize archives and atomic write failures are tested +- [x] M1–M3 golden/JSON/registry/probe regressions and local gates pass, with all local M4 evidence recorded +- [x] Native Windows/Ubuntu/macOS GitHub CI passes for the final M4 branch +- [x] One screenshot in the README, taken from a real run ### M5 — Safe remediation @@ -573,8 +612,8 @@ tests and records local measurements without claiming M6 clean-host/release cove | ENV-023A | Opt-in bounded proxy evidence collector and offline-default regression checks | ENV-005 | M3 | | ENV-022 | Rules ENV007–ENV012 | ENV-020, ENV-021, ENV-023A | M3 | | ENV-023 | Rule registry, `explain`, `rules --json` and docs generation | ENV-022 | M3 | -| ENV-024 | Bubble Tea renderer with plain fallback | ENV-019 | M4 | -| ENV-025 | Bundle writer with redaction tests | ENV-005 | M4 | +| ENV-024 | Bubble Tea/plain selection and escaped Markdown export | ENV-019 | M4 | +| ENV-025 | Versioned bounded bundle writer/reader and allowlisted redaction | ENV-005, ENV-024 | M4 | | ENV-026 | `diagnose --from-bundle` | ENV-025 | M4 | | ENV-027 | Fix planner and dry run | ENV-022 | M5 | | ENV-028 | Fix apply, logging and undo | ENV-027 | M5 | @@ -590,7 +629,7 @@ tests and records local measurements without claiming M6 clean-host/release cove - **CLI integration tests** run the built binary against a fixture directory (`--fixture-dir`), asserting stdout, stderr and exit code. - **Redaction tests** attempt to leak synthetic credentials through snapshots, findings and notices. - Bundle sharing/redaction remains a separate M4 acceptance gate. + M4 sharing tests exercise every member, fixed error output and structured diagnostics. - **Cross-platform:** the same suite runs on `windows-latest`, `ubuntu-latest`, `macos-latest`. A rule that has a platform exclusion records it in registry caveats and explicit inconclusive fixtures. @@ -605,7 +644,7 @@ No telemetry, by design. Diagnostics only: - `--verbose` prints collector timings; errors are redacted. - Planned `envdoctor diagnose --debug-snapshot ` will write a snapshot for bug reports. -- Planned bundles will include tool version, OS build and collector timings. +- Bundles include normalized tool/platform metadata, explicit omissions and optional structured collector timings. There is no `envdoctor.io` endpoint. There is no analytics. A PR that adds phone-home is rejected. @@ -654,10 +693,11 @@ Explicitly out of scope: malware detection, host compromise assessment. ## 19. Anti-hallucination contract -**Facts:** M1–M3 collection, twelve pure diagnostic rules, plain-text/JSON reporting, offline metadata -commands and bounded offline diagnosis are implemented and tested. Native Windows/Linux local race -suites and native Windows/Ubuntu/macOS M3 CI pass. Bundles, automatic fixes, rules ENV013–ENV018 -and signed releases remain future work. Runtime discovery does not execute PATH candidates and unknown +**Facts:** M1–M4 collection, twelve pure rules, terminal/plain/Markdown/JSON reporting, offline metadata +commands and allowlisted bundles/offline diagnosis are implemented and locally tested. Native Windows/ +Linux race and all six static builds pass. Native M3 and M4 three-OS GitHub CI passed. +Automatic fixes, rules ENV013–ENV018 and signed releases remain future work. Runtime discovery does not +execute PATH candidates and unknown metadata stays unknown. Optional proxy reachability requires explicit opt-in. **Rules:** @@ -678,7 +718,7 @@ metadata stays unknown. Optional proxy reachability requires explicit opt-in. | Rule | Pure function from snapshot to findings. Has a permanent id. | | Finding | One diagnosis: severity, evidence, cause, fix. | | Fix | A proposed remediation with explicit safety properties. | -| Bundle | A zip of report, snapshot and logs, safe to share publicly. | +| Bundle | Versioned export of redacted captured evidence/reports under a documented sharing policy. | | Degraded | A collector failed; its rules are skipped, not guessed. | --- @@ -688,5 +728,8 @@ metadata stays unknown. Optional proxy reachability requires explicit opt-in. 1. Read sections 0, 3 and 9. Ten minutes. 2. `make build && ./bin/envdoctor diagnose`; compare what it prints with section 0's `functional` field. 3. Pick the next unchecked item in the current milestone. Rules come with fixtures or they do not come. + M4 requirements are in [phases/M4.md](phases/M4.md); ADR 0006 is accepted. Implementation was requested + on 2026-11-19 and is verified at `c532dc3`, including native GitHub CI. Preserve accepted export/CLI + decisions; PR #4 is open for review. See [M4 validation](testing/m4-validation.md). 4. Run `make test` — golden fixtures will tell you immediately if a rule change broke another platform. 5. Never add a fix without the safety model. If in doubt, print the command instead of running it. diff --git a/docs/adr/0006-m4-reports-and-bundles.md b/docs/adr/0006-m4-reports-and-bundles.md new file mode 100644 index 0000000..dedc5b3 --- /dev/null +++ b/docs/adr/0006-m4-reports-and-bundles.md @@ -0,0 +1,141 @@ +# ADR 0006 — M4 report formats and shareable diagnostic evidence + +- Status: accepted +- Proposed: 2026-11-17; accepted: 2026-11-19 +- Required by: [M4 requirements](../phases/M4.md), ENV-024–ENV-026 +- Extends: ADRs 0004/0005 with a separate bundle export/offline-input contract + +## Context + +M3 plain/JSON reports and fixture-directory diagnosis are implemented. Ordinary JSON intentionally +retains effective Git identity. Home/credential-URL redaction is not a sharing policy. Bundles must be +smaller than 2 MB, remove identifying/raw data and preserve the meaning of absent diagnostic evidence. +Saved reports are reference data, not verdicts or instructions to execute on another machine. + +## Decision + +Implement ENV-024–ENV-026 below. Preserve collectors, twelve builtin rules, registry pages, fixture-v1 +input, ordinary JSON and diagnosis exits. No command in this phase applies a fix. + +### Formats and terminal lifecycle + +- `diagnose --format text|markdown|json` selects an explicit format. `--json` remains a JSON alias. + `--plain` forces text, as does explicit text. `--json --format json` is compatible; conflicting + formats, plain with JSON/Markdown and unknown formats exit 4 before collection. +- Without an explicit format/plain choice, use Bubble Tea only when stdin, stdout **and stderr** are + terminals, TERM is not `dumb`, and NO_COLOR is not set. Missing TERM on Windows does not preclude a + terminal. Pipes, redirects and non-file test streams fall back to stable plain output. JSON/Markdown + bypass the viewer. Verbose collector timings remain on stderr. +- Pin `github.com/charmbracelet/bubbletea v1.3.10` and `golang.org/x/term v0.48.0`; commit exact + transitive module versions/checksums and audit/review licenses before completion. +- The viewer scrolls the completed report, handles narrow dimensions/resize and uses an alternate screen. + Quit returns the diagnosis exit; interruption/render failure returns 3. Every exit restores terminal + state and prints the plain report. Output errors cancel the viewer and propagate as failures. + No viewer action executes advice, collects evidence or changes a finding. +- Text makes terminal controls visible. Markdown emits repository-owned structure and escapes dynamic + punctuation/controls, including HTML, links, fences and commands/paths. Include severity/ID, evidence, + limitations, manual advice/verification, degraded collectors and skipped prerequisites from the same + report/registry. Missing evidence is not displayed as healthy. + +### Separate sharing snapshot + +Bundle snapshots have a separate version-1 typed allowlist. Convert approved facts into existing pure +engine input; generate JSON/Markdown/versions from that evidence, never from saved finding text. + +| Captured field | Sharing representation and rule effect | +|---|---| +| Timestamp/platform | Original UTC capture time; known OS/architecture, elevation and bounded numeric version metadata. Unknown platform stays absent/degraded. | +| Shell | Known shell-name enum; path removed. Unknown names are unavailable with an omission. | +| PATH | Order/existence/directory/duplicate facts; raw locations become capture-local opaque path tokens. | +| Runtimes | Python/Node names, numeric versions, known manager/resolution enums and active/expected/missing facts. Exact path aliases preserve comparisons; separate normalized environment aliases preserve ENV011 equality under the captured OS. | +| Requested ports/listeners | Port/protocol/state facts; PID/process labels omitted. Ownership detail is unavailable while ENV005 can still detect an occupied requested port. | +| Docker | Installed/local probe/API-range/override facts and numeric versions. Selected context names/endpoint URLs omitted; they never select a connection target. | +| Git | Version, identity-checked and nonempty-name/email **presence booleans**, autocrlf policy and opaque origin markers. Reconstruct fixed presence markers for ENV010; redaction is not an unset identity. | +| Dependencies | Supported tool/library names, present/missing/unknown status, opaque paths/project/requirements markers and allowed Git policy. Keep supported package-manager names/exact versions and known lockfile names. | +| Python environment | Metadata-known and capture-local canonical environment tokens; no receiving-host canonicalization or lookup. | +| Proxies | Known variable/scheme/port/probe facts; original host/URL become capture-local `.invalid` aliases. No userinfo, hostname, path, query or fragment. Offline evaluation never resolves/dials aliases. | +| Failures/timings | Known collector identifier, bounded milliseconds and a fixed unavailable notice. Original errors/stdout/stderr/free-text logs omitted. | +| Report/tool metadata | Fixed tool name and normalized version metadata. Captured findings/advice come from the builtin registry after export, not the original report's free text. | + +Aliases are deterministic in field traversal order and local to a capture; they contain no hash or raw +path/identity. They preserve only the documented equality and are not filesystem locations. Omissions +use fixed field/reason identifiers. Removing required evidence marks that collector unavailable and +skips dependent rules under existing prerequisites. Removing non-prerequisite labels stays visible +without fabricating an error. Unknown/future export fields fail closed; unsupported values are omitted +or rejected rather than copied through. + +No raw environment values, tokens, usernames/emails, hostname, home paths, process arguments, project +contents or arbitrary logs are exported. This is a current-field allowlist, not universal secret +detection. Numeric versions/ports, platform and diagnostic facts still describe the captured setup. + +### Bundle v1 and independent bounds + +Required ZIP members: `manifest.json`, `snapshot.json`, `report.json`, `report.md`, `versions.json`. +Optional `diagnostics.json`: `bundle --include-logs` means only allowlisted structured collector +statuses/timings, never shell history, files or command output. Every member uses the sharing policy. + +The manifest identifies bundle/snapshot/report/redaction/tool versions, original capture time and +diagnosis exit, omissions, and every other member's exact size/SHA-256. It excludes its own digest. +Digests detect corruption; they do not authenticate a machine/author. The exported report exposes the +original exit as reference metadata and its recomputed exit, including omitted-prerequisite degradation. + +| Limit | Contract | +|---|---| +| Compressed archive, including directory | Fewer than **2,000,000 bytes** | +| Total decompressed members | At most **8 MiB** | +| Entries | At most eight; exactly five required names plus optional diagnostics | +| Manifest / versions | At most 64 KiB each | +| Snapshot / Markdown | At most 2 MiB each | +| JSON report / diagnostics | At most 3 MiB / 512 KiB | +| JSON work | One document; no duplicate/unknown fields; depth ≤32, tokens ≤200,000, strings ≤4,096 UTF-8 bytes; arrays ≤4,096 elements | +| Evidence collections | PATH ≤1,024; runtimes ≤128; listeners/requested ports ≤1,024; dependencies ≤32; proxies ≤3; errors ≤32; omissions ≤256 | + +Preflight file/directory limits before constructing an unbounded ZIP/JSON graph; enforce actual +streamed decompressed sizes. Accept only Store/Deflate regular entries. Reject encryption, +ZIP64/multi-disk/unsupported flags, extra/comment records, duplicate/unknown or nonliteral names, +directories/symlinks, absolute/drive/UNC/backslash/traversal paths, inconsistent local/central headers, +overlaps, size/digest/CRC mismatches and trailing data. Never extract. Validate required structures and +cross-member capture metadata before evaluation. + +An oversized capture fails explicitly without truncation, partial publication or relaxed limits. +Stage a private temporary file beside the requested destination, finish/sync/close and validate it, +then replace the named regular destination by same-directory rename (Windows replacement API). +Never follow a destination symlink. Failure/cancellation removes only the owned temporary file and +keeps the previous destination. Atomic namespace replacement assumes a supporting local filesystem; +network-filesystem semantics and power-loss durability are not guarantees of this format. + +### Commands, provenance and exits + +- `bundle --output FILE` defaults to `envdoctor-bundle.zip`; `-o` is an alias. Existing capture controls + `--fixture-dir`, `--port`, `--probe-network`, `--package-manager` and optional `--include-logs` apply. + It captures the complete builtin report; format/rule filters do not select archive members. + Successful publication exits 0 and prints bounded status/byte count, even for captured warnings/errors/ + degradation. Capture/archive/cancellation/output/I/O failures exit 3; usage exits 4. +- `diagnose --from-bundle FILE` validates and evaluates sharing evidence using the current builtin registry. + It runs no collectors, PATH/Git/Docker/project commands, home/path inspection or network work. Preserve + capture time/platform, label capture-tool/current-evaluator versions, and expose omissions. Saved + verdicts/suggestions/Markdown are reference members and cannot influence evaluation. +- Bundle input conflicts with fixture input and requested-port/network-probe/package-manager live flags. + Format/component/rule/verbose filters remain available. Fixture behavior, including its existing + requested-port selection, stays unchanged. Invalid combinations fail before I/O. +- Malformed/unsupported/bounded-out/corrupt input and read failures exit 3; usage exits 4. Valid offline + evaluation preserves 0/1/2/3, including 3 for unavailable required evidence. Bundle errors have fixed + reason text and do not interpolate untrusted fields/member names/captured errors. +- Ordinary JSON remains schema v1 with existing fields/values and Git identity. Optional bundle-source + metadata is absent from ordinary runs and documented as a compatible report-schema addition. + +## Alternatives considered + +- Zip ordinary JSON: retains identity/unreviewed free text and fails the sharing contract. +- Remove paths/identities without presence/equality/omissions: causes false missing-setting or healthy results. +- Extract ZIP or run captured commands: unnecessary filesystem/execution authority; pure evaluation suffices. +- Reuse saved verdicts: loses evaluator provenance and turns untrusted instructions into findings. + +## Consequences and validation + +Renderer/Markdown cases precede export. Add reviewed redaction/archive-abuse cases before writer/reader +and offline CLI. Verify all original 172 cases, strict schemas, deterministic cross-OS evidence, privacy +across every member/error, exact size boundaries and rejection cleanup. Measure repeated archive/viewer +lifecycles using synthetic captures. Record native terminal/Windows/Ubuntu/macOS CI, dependency/licenses +and a real interactive screenshot separately from cross-builds/model-only tests. Check acceptance boxes +only after named evidence exists. diff --git a/docs/adr/README.md b/docs/adr/README.md index ff673ea..ae1361b 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -8,7 +8,8 @@ Each file records one decision that was expensive to make and would be expensive | [0002](0002-m1-implementation-decisions.md) | M1 implementation decisions | accepted, partly superseded by 0003 and 0005 | | [0003](0003-go-126-floor-and-ci-contract.md) | Go 1.26 floor and CI contract | accepted | | [0004](0004-m2-collectors-and-rules.md) | M2 collectors and rules | accepted, scoped extensions in 0005 | -| [0005](0005-m3-evidence-and-rule-registry.md) | M3 evidence, bounded network opt-in and rule metadata | accepted for the M3 feature branch | +| [0005](0005-m3-evidence-and-rule-registry.md) | M3 evidence, bounded network opt-in and rule metadata | accepted; merged M3 | +| [0006](0006-m4-reports-and-bundles.md) | M4 report formats and shareable diagnostic evidence | accepted; implementation in progress | New ADRs are numbered sequentially and never edited after acceptance. A changed mind gets a new ADR that supersedes the old one. diff --git a/docs/images/m4-terminal.jpg b/docs/images/m4-terminal.jpg new file mode 100644 index 0000000..91d20d8 Binary files /dev/null and b/docs/images/m4-terminal.jpg differ diff --git a/docs/phases/M3.md b/docs/phases/M3.md index fb64f40..2f4ff1b 100644 --- a/docs/phases/M3.md +++ b/docs/phases/M3.md @@ -1,6 +1,8 @@ # M3 requirements: evidence-based rules and inspectors -Status: **Complete on `feat/m3-implementation`, including native Windows/Ubuntu/macOS CI.** +Status: **Complete and merged into main** through [PR #3](https://github.com/Shellghostt/EnvDoctor/pull/3), +2026-11-16. Native Windows/Ubuntu/macOS CI and +[main CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43802280915) passed. Prepared 2026-11-14 from merged M2 commit `c7e2c84c730edca814a44580cdfc9abaf630b457`. The [plan](../PLAN.md) defines scope; this is its M3 acceptance contract. @@ -86,7 +88,8 @@ The [M1/M2 record](../testing/m1-m2-validation.md) preserves historical results ## Handoff -ENV-020–ENV-023A and all M3 acceptance gates are verified on the feature branch. The recorded CI run +ENV-020–ENV-023A and all M3 acceptance gates are verified and merged. The recorded feature-branch CI run verifies M3 source at `fbf79dd`, including native macOS execution. Follow [development tools](../DEVELOPMENT-TOOLS.md) and provide the branch, fixture/reproducer, exact command and redacted output for debugging in another editor. +Planned report/bundle work is defined by [M4 requirements](M4.md), with ADR 0006 still proposed. diff --git a/docs/phases/M4.md b/docs/phases/M4.md new file mode 100644 index 0000000..520aecc --- /dev/null +++ b/docs/phases/M4.md @@ -0,0 +1,142 @@ +# M4 requirements: terminal reports and diagnostic bundles + +Status: **Implemented and verified on `feat/m4-implementation`, 2026-11-19.** +Implementation: `c532dc3`. Native M4 GitHub CI passed; +[PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) is open for review. See +[M4 validation](../testing/m4-validation.md) for executed evidence and limits. +Baseline: merged M3 main commit `6cf358138ecd3bedf086f22536a30562fb673e4e`, +[PR #3](https://github.com/Shellghostt/EnvDoctor/pull/3), with successful +[main CI](https://github.com/Shellghostt/EnvDoctor/actions/runs/43802280915). +Read the [plan](../PLAN.md), [architecture](../ARCHITECTURE.md), [M3 requirements](M3.md) and +accepted [ADR 0005](../adr/0005-m3-evidence-and-rule-registry.md). +[ADR 0006](../adr/0006-m4-reports-and-bundles.md) was accepted before dependent implementation. + +## Scope and compatibility + +Implement ENV-024–ENV-026: Bubble Tea with a plain fallback, Markdown export, a redacted bundle writer, +and `diagnose --from-bundle`. Include Markdown rendering explicitly in ENV-024; it currently has no +separate backlog item. Rules ENV013–ENV018, fix/apply/undo and signed packaging remain later milestones. + +Preserve the nine collectors, twelve pure rules, immutable registry, evidence prerequisites, generated +rule pages, schema-v1 fixture support, deterministic finding ordering and existing JSON/exit behavior. +Ordinary `--json` still writes one report document to stdout; diagnostics/progress belong on stderr. +Diagnosis applies no fixes. Default collection remains offline, and proxy probing remains an explicit, +bounded live opt-in. No export or offline input enables a command supplied by a bundle. + +## Renderer contract + +- Interactive output is selected only when the required streams are terminals and terminal capabilities + permit it. Define explicit plain-output/format flags, precedence and invalid combinations in ADR 0006. + A redirect, pipe, `TERM=dumb`, `NO_COLOR` or explicit plain choice must have a documented fallback. +- Non-TTY/default plain output remains stable and contains no ANSI/cursor-control codes. JSON and + Markdown bypass Bubble Tea, even in a terminal. Findings/collector failures retain their exit codes. +- Markdown includes severity, rule ID, evidence, limitations, manual suggestions and verification text + from the same report/registry. Escape hostile labels, pipes, HTML, links and backtick/fence sequences; + never turn captured commands or paths into executable behavior or active content. +- Do not present missing/denied/redacted evidence as a healthy result. Show degraded collectors and + skipped rules alongside findings. Export progress/timings must not corrupt stdout formats. +- Resize, cancellation and output failure must restore terminal state and release timers/goroutines. + An interactive rendering failure must have an explicit failure/fallback path, not discard the report. + +## Bundle format and bounds + +The zip is a versioned sharing format, separate from the existing fixture-directory snapshot contract. +Accepted v1 members are `manifest.json`, `snapshot.json`, `report.json`, `report.md`, `versions.json` and +optional structured `diagnostics.json`. A manifest records bundle/schema/tool versions, original capture +time, redaction policy, omitted evidence and member names/sizes/digests. Digests detect corruption; +they do not authenticate the machine or its author. + +Retain the PLAN's under-2-MB requirement: define this as **fewer than 2,000,000 compressed bytes**. +ADR 0006 also limits total uncompressed member bytes to 8 MiB and entries to eight, with documented +per-member, JSON-depth/token and string bounds. Enforce limits before allocation/decompression where +possible and while streaming. A too-large capture fails explicitly; no partial archive or silently +truncated evidence is published. Stage output locally and replace it atomically after successful checks. + +Read archives as data without extraction. Reject unsupported versions/methods, encrypted or ZIP64 input +unless explicitly supported, duplicate/unknown member names, directory/symlink entries, absolute paths, +drive/UNC names, backslash paths and traversal. Bound compressed data, directory metadata, decompressed +bytes and parse work independently. Verify member integrity and required schemas before diagnosis. +Never follow member paths, load a plugin, execute a verification/fix command or fetch a URL from a bundle. + +## Export/redaction policy + +Today's report redaction removes home paths and credential-bearing URLs, but retains effective Git +identity. Copying an existing JSON report is not a sharing policy. Define a separate allowlisted export +model before implementation, including the treatment of each field needed by ENV001–ENV012. + +- Exclude raw environment values, tokens, usernames/emails, hostname, original home paths, process + arguments, project file contents and arbitrary logs. Tool versions may be normalized metadata only. +- Sanitize or omit free text, paths, URL components, errors, notices, hints and report metadata before + writing any member. Build report JSON/Markdown from sanitized evidence, rather than embedding the + original report and relying on a final string-replacement pass. +- Preserve diagnostic distinctions where possible with nonidentifying facts or capture-local opaque + replacements. If removing an identity/path loses required evidence, mark it omitted and skip or report + the dependent rule as inconclusive; absence caused by redaction is not proof of a missing setting. +- Apply the same policy to optional tool diagnostics. `--include-logs` must not collect shell history, + arbitrary files or unstructured command output; specify its allowlisted meaning or leave it unsupported. +- Test synthetic secrets/identities through every exported member, entry name, error path and stdout/ + stderr, including encoded URLs, Windows/Unix paths and degraded collectors. An unsupported field + cannot fall back to raw export. Describe the actual policy and its limits; do not promise universal + secret detection or that every arbitrary future string is safe to publish. + +## Offline diagnosis and CLI behavior + +`diagnose --from-bundle FILE` loads validated exported evidence and evaluates the current builtin registry +without running collectors, git/Docker, PATH tools, project scripts or network probes. Do not inspect +the receiving host to fill capture gaps. Retain original capture time/platform and identify captured +evidence, current evaluator version and omissions. Captured findings are reference data; recomputed +findings come from validated evidence, not the saved report's verdict or instructions. + +Keep `--fixture-dir` unchanged. Define mutual exclusion with `--from-bundle` and live-evidence arguments +(`--port`, `--probe-network`, `--package-manager`). Output/evaluation filters may remain available. +Invalid usage exits 4; malformed/unsupported/bounded-out bundles or I/O failure exit 3. Valid offline +diagnosis uses the existing finding/degraded exit mapping 0/1/2/3, including redaction-related omissions. +The writer exits 0 when it successfully publishes an archive. The original diagnosis exit remains +reference metadata; the exported report records its recomputed exit, including omitted prerequisites. + +## Dependency order + +| Task | Work | Prerequisites | +|---|---|---| +| ENV-024 | Renderer selection, plain fallback, Bubble Tea and Markdown | Accepted format/TTY/exit policy; M3 report/registry | +| ENV-025 | Versioned allowlisted export, bounded zip writer/reader and redaction cases | Accepted bundle schema/security policy; ENV-024 report formats | +| ENV-026 | Offline `--from-bundle`, source/omission reporting and CLI tests | ENV-025; accepted evidence/flag/exit semantics | + +## Acceptance evidence + +- [x] Non-TTY, redirected stdout/stderr, dumb terminals, `NO_COLOR`, explicit plain, JSON and Markdown + combinations have deterministic CLI tests; structured output contains no ANSI/progress contamination. +- [x] A real interactive run handles narrow width, resize, interrupt and output failure without leaving + the terminal altered or retaining goroutines/timers after exit. +- [x] Markdown goldens cover every severity, collector errors, omitted evidence, hostile HTML/links, + pipes/backticks and long paths, without changing finding order or severity. +- [x] Every successful bundle is below 2,000,000 compressed bytes; exact boundary/oversize cases, + atomic output failure and independent decompression/entry/parse limits are tested. +- [x] The export policy removes synthetic tokens, raw environment values and identities from all members + and diagnostics, including Git identity, errors and optional logs; unknown export fields fail closed. +- [x] Malformed, corrupt, duplicate, traversal/symlink, compressed-bomb and unsupported-version archives + are rejected without extraction or unbounded memory growth. +- [x] Windows/Linux/macOS captures can be diagnosed on another OS offline; retained evidence reproduces + applicable results while redacted prerequisites remain explicitly inconclusive. +- [x] Offline diagnosis executes no collectors/commands/network calls, ignores saved verdict instructions, + distinguishes original capture/evaluator versions and rejects conflicting live-evidence flags. +- [x] Existing 172 golden cases, M3 JSON/schema/exit, registry generation, purity, probe and local race gates pass. +- [x] Terminal dependencies are pinned/audited; canonical local build/test/lint/race/docs gates pass. +- [x] Native Windows/Ubuntu/macOS GitHub CI passes for the final M4 branch. +- [x] One README screenshot is captured from the implemented interactive CLI with synthetic identities. +- [x] The M4 validation record maps every criterion to executed cases, commands, limits and skips. + +Use existing `make test`, `make race`, `make lint`, `make build` and `make check-docs`, with Go 1.26.11. +Native Windows race needs the x64 GCC configuration in [M3 validation](../testing/m3-validation.md). +Archive/redaction/renderer/CLI cases are mapped in [M4 validation](../testing/m4-validation.md). +Repeated bundle read/write, cancellation and renderer lifecycles were measured with synthetic captures; +read-only Windows diagnosis stayed within the local time/memory targets. Native Windows/Ubuntu/macOS +race, tests and builds passed in GitHub CI, separately from cross-builds and cross-OS evidence tests. +Native macOS interactive/performance observation and clean-host/release checks remain later work. + +## Handoff + +First settle the sharing schema and omissions, then the renderer/CLI flags; add reviewed redaction and +archive abuse fixtures before the writer/reader. Preserve accepted ADRs and generated rule pages. +M4 implementation was requested on 2026-11-19 and is verified at `c532dc3`, including native GitHub CI. +The published branch is in PR #4 for review; merging remains a separate decision. diff --git a/docs/schema/bundle-common-v1.schema.json b/docs/schema/bundle-common-v1.schema.json new file mode 100644 index 0000000..be964ae --- /dev/null +++ b/docs/schema/bundle-common-v1.schema.json @@ -0,0 +1,238 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json", + "title": "EnvDoctor bundle v1 common types", + "$defs": { + "version": { + "type": "string", + "pattern": "^(?:[0-9]{1,9}(?:\\.[0-9]{1,9}){0,3})?$", + "maxLength": 39 + }, + "tool": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version" + ], + "properties": { + "name": { + "const": "envdoctor" + }, + "version": { + "type": "string", + "pattern": "^(?:dev|[0-9]{1,9}(?:\\.[0-9]{1,9}){0,3})$", + "maxLength": 39 + } + } + }, + "utc": { + "type": "string", + "format": "date-time", + "pattern": "Z$", + "maxLength": 40 + }, + "collector": { + "type": "string", + "enum": [ + "platform", + "shell", + "path", + "runtimes", + "network", + "docker", + "git", + "deps", + "proxy" + ] + }, + "omission": { + "type": "object", + "additionalProperties": false, + "required": [ + "field", + "reason" + ], + "properties": { + "field": { + "type": "string", + "enum": [ + "platform", + "platform.version", + "shell.name", + "shell.path", + "path.locations", + "runtimes.paths", + "runtimes.version", + "runtimes.name", + "runtimes.manager", + "runtimes.resolution", + "runtimes.environment", + "network.ownership", + "docker.context", + "docker.endpoint", + "docker.version", + "docker.api", + "git.identity", + "git.origins", + "git.version", + "git.autocrlf", + "dependencies.locations", + "dependencies.requirement", + "dependencies.packageManager", + "dependencies.lockfile", + "dependencies.gitAutocrlf", + "pythonEnvironment.locations", + "proxy.endpoint", + "proxy.url", + "collector.errors", + "tool.version" + ] + }, + "reason": { + "type": "string", + "enum": [ + "identifying_values", + "raw_error", + "unsupported_value", + "unavailable_evidence" + ] + } + } + }, + "source": { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "captureTool", + "capturedAt", + "redactionPolicy", + "captureExitCode", + "omissions" + ], + "properties": { + "kind": { + "const": "bundle" + }, + "captureTool": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version" + ], + "properties": { + "name": { + "const": "envdoctor" + }, + "version": { + "type": "string", + "pattern": "^(?:dev|[0-9]{1,9}(?:\\.[0-9]{1,9}){0,3})$", + "maxLength": 39 + } + } + }, + "capturedAt": { + "type": "string", + "format": "date-time", + "pattern": "Z$", + "maxLength": 40 + }, + "redactionPolicy": { + "const": "allowlist-v1" + }, + "captureExitCode": { + "type": "integer", + "minimum": 0, + "maximum": 3 + }, + "omissions": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "field", + "reason" + ], + "properties": { + "field": { + "type": "string", + "enum": [ + "platform", + "platform.version", + "shell.name", + "shell.path", + "path.locations", + "runtimes.paths", + "runtimes.version", + "runtimes.name", + "runtimes.manager", + "runtimes.resolution", + "runtimes.environment", + "network.ownership", + "docker.context", + "docker.endpoint", + "docker.version", + "docker.api", + "git.identity", + "git.origins", + "git.version", + "git.autocrlf", + "dependencies.locations", + "dependencies.requirement", + "dependencies.packageManager", + "dependencies.lockfile", + "dependencies.gitAutocrlf", + "pythonEnvironment.locations", + "proxy.endpoint", + "proxy.url", + "collector.errors", + "tool.version" + ] + }, + "reason": { + "type": "string", + "enum": [ + "identifying_values", + "raw_error", + "unsupported_value", + "unavailable_evidence" + ] + } + } + }, + "maxItems": 256, + "uniqueItems": true + } + } + }, + "pathToken": { + "type": "string", + "pattern": "^path-(?:000[1-9]|00[1-9][0-9]|0[1-9][0-9]{2}|[1-3][0-9]{3}|40[0-8][0-9]|409[0-6])$" + }, + "environmentToken": { + "type": "string", + "pattern": "^env-(?:000[1-9]|00[1-9][0-9]|0[1-9][0-9]{2}|[1-3][0-9]{3}|40[0-8][0-9]|409[0-6])$" + }, + "hostToken": { + "type": "string", + "pattern": "^host-(?:000[1-9]|00[1-9][0-9]|0[1-9][0-9]{2}|[1-3][0-9]{3}|40[0-8][0-9]|409[0-6])\\.invalid$" + }, + "api": { + "type": "string", + "pattern": "^(?:1\\.[0-9]{1,6})?$", + "maxLength": 8 + }, + "autocrlf": { + "type": "string", + "enum": [ + "", + "true", + "false", + "input" + ] + } + } +} diff --git a/docs/schema/bundle-diagnostics-v1.schema.json b/docs/schema/bundle-diagnostics-v1.schema.json new file mode 100644 index 0000000..ffee31a --- /dev/null +++ b/docs/schema/bundle-diagnostics-v1.schema.json @@ -0,0 +1,46 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/shellghostt/envdoctor/schema/bundle-diagnostics-v1.schema.json", + "title": "EnvDoctor allowlisted collector diagnostics v1", + "type": "object", + "additionalProperties": false, + "required": [ + "schemaVersion", + "collectors" + ], + "properties": { + "schemaVersion": { + "const": 1 + }, + "collectors": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "collector", + "durationMs", + "status" + ], + "properties": { + "collector": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/collector" + }, + "durationMs": { + "type": "integer", + "minimum": 0, + "maximum": 60000 + }, + "status": { + "type": "string", + "enum": [ + "available", + "unavailable" + ] + } + } + }, + "maxItems": 32 + } + } +} diff --git a/docs/schema/bundle-manifest-v1.schema.json b/docs/schema/bundle-manifest-v1.schema.json new file mode 100644 index 0000000..d805363 --- /dev/null +++ b/docs/schema/bundle-manifest-v1.schema.json @@ -0,0 +1,85 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/shellghostt/envdoctor/schema/bundle-manifest-v1.schema.json", + "title": "EnvDoctor ZIP manifest v1", + "type": "object", + "additionalProperties": false, + "required": [ + "bundleVersion", + "snapshotVersion", + "reportVersion", + "redactionPolicy", + "tool", + "capturedAt", + "captureExitCode", + "omissions", + "members" + ], + "properties": { + "bundleVersion": { + "const": 1 + }, + "snapshotVersion": { + "const": 1 + }, + "reportVersion": { + "const": 1 + }, + "redactionPolicy": { + "const": "allowlist-v1" + }, + "tool": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/tool" + }, + "capturedAt": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/utc" + }, + "captureExitCode": { + "type": "integer", + "minimum": 0, + "maximum": 3 + }, + "omissions": { + "type": "array", + "items": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/omission" + }, + "maxItems": 256, + "uniqueItems": true + }, + "members": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "size", + "sha256" + ], + "properties": { + "name": { + "type": "string", + "enum": [ + "snapshot.json", + "report.json", + "report.md", + "versions.json", + "diagnostics.json" + ] + }, + "size": { + "type": "integer", + "minimum": 0, + "maximum": 3145728 + }, + "sha256": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + } + } + }, + "maxItems": 7 + } + } +} diff --git a/docs/schema/bundle-report-v1.schema.json b/docs/schema/bundle-report-v1.schema.json new file mode 100644 index 0000000..0e47a04 --- /dev/null +++ b/docs/schema/bundle-report-v1.schema.json @@ -0,0 +1,37 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/shellghostt/envdoctor/schema/bundle-report-v1.schema.json", + "title": "EnvDoctor shared reference diagnosis v1", + "allOf": [ + { + "$ref": "https://github.com/shellghostt/envdoctor/schema/diagnose-v1.schema.json" + }, + { + "type": "object", + "required": [ + "source" + ], + "properties": { + "source": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/source" + }, + "generatedAt": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/utc" + }, + "tool": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/tool" + }, + "findings": { + "type": "array", + "items": {}, + "maxItems": 4096 + }, + "collectors": { + "type": "array", + "items": {}, + "maxItems": 32 + } + } + } + ] +} diff --git a/docs/schema/bundle-snapshot-v1.schema.json b/docs/schema/bundle-snapshot-v1.schema.json new file mode 100644 index 0000000..b0db9f0 --- /dev/null +++ b/docs/schema/bundle-snapshot-v1.schema.json @@ -0,0 +1,627 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/shellghostt/envdoctor/schema/bundle-snapshot-v1.schema.json", + "title": "EnvDoctor allowlisted sharing snapshot v1", + "type": "object", + "additionalProperties": false, + "required": [ + "schemaVersion", + "collectedAt" + ], + "properties": { + "schemaVersion": { + "const": 1 + }, + "collectedAt": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/utc" + }, + "platform": { + "type": "object", + "additionalProperties": false, + "required": [ + "os", + "version", + "arch", + "elevated" + ], + "properties": { + "os": { + "type": "string", + "enum": [ + "windows", + "linux", + "darwin" + ] + }, + "version": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + }, + "arch": { + "type": "string", + "enum": [ + "amd64", + "arm64", + "386", + "arm", + "ppc64le", + "s390x", + "riscv64" + ] + }, + "elevated": { + "type": "boolean" + } + } + }, + "shell": { + "type": "object", + "additionalProperties": false, + "required": [ + "name" + ], + "properties": { + "name": { + "type": "string", + "enum": [ + "powershell", + "pwsh", + "cmd", + "bash", + "zsh", + "fish", + "sh", + "dash", + "ksh", + "tcsh", + "csh", + "nu" + ] + } + } + }, + "path": { + "type": "object", + "additionalProperties": false, + "required": [ + "entries" + ], + "properties": { + "entries": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "token", + "exists", + "isDir", + "position", + "duplicate" + ], + "properties": { + "token": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/pathToken" + }, + "exists": { + "type": "boolean" + }, + "isDir": { + "type": "boolean" + }, + "position": { + "type": "integer", + "minimum": 0, + "maximum": 1023 + }, + "duplicate": { + "type": "boolean" + } + } + }, + "maxItems": 1024 + } + } + }, + "runtimes": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version", + "pathToken", + "manager", + "active" + ], + "properties": { + "name": { + "type": "string", + "enum": [ + "python", + "node" + ] + }, + "version": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + }, + "pathToken": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/pathToken" + }, + "manager": { + "type": "string", + "enum": [ + "system", + "venv", + "pyenv", + "nvm", + "fnm", + "volta" + ] + }, + "active": { + "type": "boolean" + }, + "expected": { + "type": "boolean" + }, + "missing": { + "type": "boolean" + }, + "resolvedToken": { + "anyOf": [ + { "const": "" }, + { "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/pathToken" } + ] + }, + "resolution": { + "type": "string", + "enum": [ + "", + "known", + "unknown" + ] + }, + "environmentToken": { + "anyOf": [ + { "const": "" }, + { "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/environmentToken" } + ] + } + } + }, + "maxItems": 128 + }, + "ports": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "port", + "protocol", + "state" + ], + "properties": { + "port": { + "type": "integer", + "minimum": 1, + "maximum": 65535 + }, + "protocol": { + "type": "string", + "enum": [ + "tcp", + "udp" + ] + }, + "state": { + "type": "string", + "enum": [ + "listen", + "established", + "closed", + "unknown" + ] + } + } + }, + "maxItems": 1024 + }, + "requestedPorts": { + "type": "array", + "items": { + "type": "integer", + "minimum": 1, + "maximum": 65535 + }, + "maxItems": 1024 + }, + "docker": { + "type": "object", + "additionalProperties": false, + "required": [ + "cliVersion", + "installed", + "daemonReachable" + ], + "properties": { + "cliVersion": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + }, + "installed": { + "type": "boolean" + }, + "daemonReachable": { + "type": "boolean" + }, + "probeStatus": { + "type": "string", + "enum": [ + "", + "absent", + "available", + "unavailable", + "unknown", + "denied", + "unsupported" + ] + }, + "clientApi": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/api" + }, + "clientMinApi": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/api" + }, + "clientDefaultApi": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/api" + }, + "serverApi": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/api" + }, + "serverMinApi": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/api" + }, + "apiOverride": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/api" + } + } + }, + "git": { + "type": "object", + "additionalProperties": false, + "required": [ + "version", + "userNamePresent", + "userEmailPresent", + "identityChecked", + "autocrlf", + "autocrlfOriginKnown" + ], + "properties": { + "version": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + }, + "userNamePresent": { + "type": "boolean" + }, + "userEmailPresent": { + "type": "boolean" + }, + "identityChecked": { + "type": "boolean" + }, + "autocrlf": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/autocrlf" + }, + "autocrlfOriginKnown": { + "type": "boolean" + } + } + }, + "dependencies": { + "type": "object", + "additionalProperties": false, + "required": [ + "projectToken", + "requirementsDeclared" + ], + "properties": { + "projectToken": { + "anyOf": [ + { + "const": "" + }, + { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/pathToken" + } + ] + }, + "requirementsDeclared": { + "type": "boolean" + }, + "requirements": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "name", + "status" + ], + "properties": { + "kind": { + "type": "string", + "enum": [ + "tool", + "library" + ] + }, + "name": { + "type": "string", + "enum": [ + "cmake", + "make", + "gcc", + "pkg-config", + "openssl" + ] + }, + "pathToken": { + "anyOf": [ + { "const": "" }, + { "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/pathToken" } + ] + }, + "status": { + "type": "string", + "enum": [ + "present", + "missing", + "unknown" + ] + } + }, + "allOf": [ + { + "if": { + "properties": { + "kind": { + "const": "tool" + } + } + }, + "then": { + "properties": { + "name": { + "type": "string", + "enum": [ + "cmake", + "make", + "gcc", + "pkg-config" + ] + } + } + } + }, + { + "if": { + "properties": { + "kind": { + "const": "library" + } + } + }, + "then": { + "properties": { + "name": { + "const": "openssl" + } + } + } + } + ] + }, + "maxItems": 32 + }, + "package": { + "type": "object", + "additionalProperties": false, + "required": [ + "manifestPresent" + ], + "properties": { + "manifestPresent": { + "type": "boolean" + }, + "declared": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version" + ], + "properties": { + "name": { + "type": "string", + "enum": [ + "npm", + "pnpm", + "yarn", + "bun" + ] + }, + "version": { + "type": "string", + "pattern": "^(?:0|[1-9][0-9]{0,8})\\.(?:0|[1-9][0-9]{0,8})\\.(?:0|[1-9][0-9]{0,8})$" + } + } + }, + "runner": { + "type": "object", + "additionalProperties": false, + "required": [ + "name" + ], + "properties": { + "name": { + "type": "string", + "enum": [ + "npm", + "pnpm", + "yarn", + "bun" + ] + }, + "version": { + "type": "string", + "pattern": "^(?:0|[1-9][0-9]{0,8})\\.(?:0|[1-9][0-9]{0,8})\\.(?:0|[1-9][0-9]{0,8})$" + } + } + }, + "lockfiles": { + "type": "array", + "items": { + "type": "string", + "enum": [ + "package-lock.json", + "npm-shrinkwrap.json", + "pnpm-lock.yaml", + "yarn.lock", + "bun.lock", + "bun.lockb" + ] + }, + "maxItems": 4096 + } + } + }, + "gitAutocrlf": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/autocrlf" + } + } + }, + "pythonEnvironment": { + "type": "object", + "additionalProperties": false, + "required": [ + "rootToken", + "metadataKnown" + ], + "properties": { + "rootToken": { + "anyOf": [ + { + "const": "" + }, + { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/environmentToken" + } + ] + }, + "canonicalToken": { + "anyOf": [ + { "const": "" }, + { "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/environmentToken" } + ] + }, + "metadataKnown": { + "type": "boolean" + } + }, + "allOf": [ + { + "if": { + "properties": { + "metadataKnown": { + "const": true + } + } + }, + "then": { + "required": [ + "canonicalToken" + ] + } + } + ] + }, + "proxies": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "variable", + "scheme", + "hostToken", + "port", + "probe" + ], + "properties": { + "variable": { + "type": "string", + "enum": [ + "HTTP_PROXY", + "HTTPS_PROXY", + "ALL_PROXY" + ] + }, + "scheme": { + "type": "string", + "enum": [ + "http", + "https", + "socks5", + "socks5h" + ] + }, + "hostToken": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/hostToken" + }, + "port": { + "type": "integer", + "minimum": 1, + "maximum": 65535 + }, + "probe": { + "type": "string", + "enum": [ + "untested", + "reachable", + "unreachable", + "unknown", + "invalid" + ] + } + } + }, + "maxItems": 3 + }, + "noProxyConfigured": { + "type": "boolean" + }, + "errors": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "collector", + "reason" + ], + "properties": { + "collector": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/collector" + }, + "reason": { + "const": "unavailable" + } + } + }, + "maxItems": 32 + } + } +} diff --git a/docs/schema/bundle-versions-v1.schema.json b/docs/schema/bundle-versions-v1.schema.json new file mode 100644 index 0000000..18011ce --- /dev/null +++ b/docs/schema/bundle-versions-v1.schema.json @@ -0,0 +1,50 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/shellghostt/envdoctor/schema/bundle-versions-v1.schema.json", + "title": "EnvDoctor shared version metadata v1", + "type": "object", + "additionalProperties": false, + "required": [ + "schemaVersion", + "tool", + "runtimes" + ], + "properties": { + "schemaVersion": { + "const": 1 + }, + "tool": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/tool" + }, + "runtimes": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version" + ], + "properties": { + "name": { + "type": "string", + "enum": [ + "python", + "node" + ] + }, + "version": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + } + } + }, + "maxItems": 128 + }, + "git": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + }, + "docker": { + "$ref": "https://github.com/shellghostt/envdoctor/schema/bundle-common-v1.schema.json#/$defs/version" + } + } +} diff --git a/docs/schema/diagnose-v1.schema.json b/docs/schema/diagnose-v1.schema.json index ad0ee38..76bacfa 100644 --- a/docs/schema/diagnose-v1.schema.json +++ b/docs/schema/diagnose-v1.schema.json @@ -2,7 +2,7 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/shellghostt/envdoctor/schema/diagnose-v1.schema.json", "title": "EnvDoctor diagnose output v1", - "description": "The single JSON document written by `envdoctor diagnose --json`.", + "description": "The single JSON diagnosis document. Optional bundle source metadata is absent from ordinary live/fixture runs; existing v1 fields and values remain unchanged.", "type": "object", "additionalProperties": false, "required": [ @@ -17,6 +17,7 @@ "collectors" ], "properties": { + "source": {"$ref":"#/$defs/source"}, "schemaVersion": { "const": 1 }, "generatedAt": { "type": "string", "format": "date-time" }, "tool": { @@ -66,6 +67,114 @@ } }, "$defs": { + "source": { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "captureTool", + "capturedAt", + "redactionPolicy", + "captureExitCode", + "omissions" + ], + "properties": { + "kind": { + "const": "bundle" + }, + "captureTool": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version" + ], + "properties": { + "name": { + "const": "envdoctor" + }, + "version": { + "type": "string", + "pattern": "^(?:dev|[0-9]{1,9}(?:\\.[0-9]{1,9}){0,3})$", + "maxLength": 39 + } + } + }, + "capturedAt": { + "type": "string", + "format": "date-time", + "pattern": "Z$", + "maxLength": 40 + }, + "redactionPolicy": { + "const": "allowlist-v1" + }, + "captureExitCode": { + "type": "integer", + "minimum": 0, + "maximum": 3 + }, + "omissions": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "field", + "reason" + ], + "properties": { + "field": { + "type": "string", + "enum": [ + "platform", + "platform.version", + "shell.name", + "shell.path", + "path.locations", + "runtimes.paths", + "runtimes.version", + "runtimes.name", + "runtimes.manager", + "runtimes.resolution", + "runtimes.environment", + "network.ownership", + "docker.context", + "docker.endpoint", + "docker.version", + "docker.api", + "git.identity", + "git.origins", + "git.version", + "git.autocrlf", + "dependencies.locations", + "dependencies.requirement", + "dependencies.packageManager", + "dependencies.lockfile", + "dependencies.gitAutocrlf", + "pythonEnvironment.locations", + "proxy.endpoint", + "proxy.url", + "collector.errors", + "tool.version" + ] + }, + "reason": { + "type": "string", + "enum": [ + "identifying_values", + "raw_error", + "unsupported_value", + "unavailable_evidence" + ] + } + } + }, + "maxItems": 256, + "uniqueItems": true + } + } + }, "finding": { "type": "object", "additionalProperties": false, diff --git a/docs/testing/fixtures/m4-viewer/snapshot.json b/docs/testing/fixtures/m4-viewer/snapshot.json new file mode 100644 index 0000000..060695a --- /dev/null +++ b/docs/testing/fixtures/m4-viewer/snapshot.json @@ -0,0 +1,21 @@ +{ + "schemaVersion": 1, + "collectedAt": "2026-11-19T10:00:00Z", + "platform": {"os": "windows", "version": "11", "arch": "amd64", "elevated": false}, + "shell": {"name": "powershell"}, + "path": {"entries": [ + {"raw": "C:\\Synthetic\\bin", "exists": true, "isDir": true, "position": 0, "duplicate": false}, + {"raw": "C:\\Synthetic\\old-tools", "exists": false, "isDir": false, "position": 1, "duplicate": false} + ]}, + "runtimes": [ + {"name": "python", "version": "3.12.1", "path": "C:\\Synthetic\\Python312\\python.exe", "manager": "system", "active": true, "resolvedPath": "C:\\Synthetic\\Python312\\python.exe", "resolution": "known"}, + {"name": "python", "version": "3.11.9", "path": "C:\\Synthetic\\Python311\\python.exe", "manager": "system", "active": true, "resolvedPath": "C:\\Synthetic\\Python311\\python.exe", "resolution": "known"}, + {"name": "python", "version": "3.12.1", "path": "C:\\Synthetic\\project\\.venv\\Scripts\\python.exe", "manager": "venv", "active": false, "expected": true, "resolvedPath": "C:\\Synthetic\\Python312\\python.exe", "resolution": "known", "environmentRoot": "C:\\Synthetic\\project\\.venv"} + ], + "requestedPorts": [3000], + "ports": [{"port": 3000, "protocol": "tcp", "state": "listen", "process": "synthetic-listener", "pid": 42}], + "git": {"version": "2.53.0", "userName": "Synthetic Fixture", "userEmail": "fixture@example.invalid", "autocrlf": "input", "identityChecked": true}, + "docker": {"cliVersion": "29.9.2", "installed": true, "daemonReachable": true, "context": "local-default", "probeStatus": "available", "clientApi": "1.56", "clientMinApi": "1.40", "serverApi": "1.56", "serverMinApi": "1.40"}, + "dependencies": {"projectRoot": "C:\\Synthetic\\project", "package": {"manifest": "C:\\Synthetic\\project\\package.json", "declared": {"name": "pnpm", "version": "9.0.0"}, "runner": {"name": "npm", "version": "10.0.0"}, "lockfiles": ["pnpm-lock.yaml"]}}, + "pythonEnvironment": {"root": "C:\\Synthetic\\project\\.venv", "canonicalRoot": "C:\\Synthetic\\project\\.venv", "metadataKnown": true} +} diff --git a/docs/testing/m4-dependencies.md b/docs/testing/m4-dependencies.md new file mode 100644 index 0000000..a6aa1f8 --- /dev/null +++ b/docs/testing/m4-dependencies.md @@ -0,0 +1,57 @@ +# EnvDoctor M4 dependency review — 2026-11-19 + +The Go 1.26.11 module graph was read with `go list -m -json all`. Exact versions and +checksums are committed in `go.mod`/`go.sum`; `go mod verify` passed and +`govulncheck ./...` reported **No vulnerabilities found** on the final M4 implementation. + +Bubble Tea v1.3.10 and `golang.org/x/term` v0.48.0 follow accepted ADR 0006. +`golang.org/x/sys` v0.50.0, `github.com/charmbracelet/x/ansi` v0.10.1 and +`golang.org/x/text` v0.44.0 are also pinned. The binary still builds with CGO disabled. + +The reviewed graph uses MIT, BSD-3-Clause and the existing Apache-2.0 dependencies. +`github.com/mattn/go-localereader` v0.0.1 has no standalone LICENSE file in its +downloaded module. Its pinned README explicitly declares **MIT** and names Yasuhiro +Matsumoto as the author; that README, rather than a nonexistent LICENSE, is the +evidence recorded below. These are source declarations reviewed for this dependency +set, not a claim that all upstream modules ship identical license files. + +The table records SHA-256 of the exact source file read from the Go module cache. +Pseudo-versions identify their exact revisions. The existing schema test dependency +and other modules in the selected graph are included even when absent from the final +binary dependency list. + +| Module | Version | Declared license | Evidence file | SHA-256 | +|---|---|---|---|---| +| github.com/aymanbagabas/go-osc52/v2 | v2.0.1 | MIT | LICENSE | 11319e34d9dac2f45392f14f4f62e28b24173cfa13d39b32e7b7b66c2f3bdccd | +| github.com/charmbracelet/bubbletea | v1.3.10 | MIT | LICENSE | 69793670485e4ae8bf28fcbe5639141f604256d5bb6c1862571f3a32823fa221 | +| github.com/charmbracelet/colorprofile | v0.2.3-0.20250311203215-f60798e515dc | MIT | LICENSE | 57d6eeeb336030b89c0e0d12cf869f84b97a481cd00b350ef21fca995da5a39a | +| github.com/charmbracelet/lipgloss | v1.1.0 | MIT | LICENSE | 699cffa88b62691ea09706d356e5ee0982e87ca2b1c58ca682f27d6ae87f9efc | +| github.com/charmbracelet/x/ansi | v0.10.1 | MIT | LICENSE | 0baa4a5942a801d93f4050d2bc0039d8c037105fcd335230d118eec99a11782b | +| github.com/charmbracelet/x/cellbuf | v0.0.13-0.20250311204145-2c3ea96c31dd | MIT | LICENSE | 0baa4a5942a801d93f4050d2bc0039d8c037105fcd335230d118eec99a11782b | +| github.com/charmbracelet/x/term | v0.2.1 | MIT | LICENSE | 0baa4a5942a801d93f4050d2bc0039d8c037105fcd335230d118eec99a11782b | +| github.com/erikgeiser/coninput | v0.0.0-20211004153227-1c3628e74d0f | MIT | LICENSE | 0a00deeeb36bb7b8ccbb9003100b665e584d9384e66c8c26f29b5929ecaa745b | +| github.com/inconshreveable/mousetrap | v1.1.0 | Apache-2.0 | LICENSE | 4819582701f150b28a563a6cd8ed0bf5a754e3c67b90ad38d78ba4131bf77795 | +| github.com/lucasb-eyer/go-colorful | v1.2.0 | MIT | LICENSE | 1e4cf2a117516ec1b54041d68ed4f5d331878b04de3386f4790e7b5b9c04e1ec | +| github.com/mattn/go-isatty | v0.0.20 | MIT | LICENSE | 08eab1118c80885fa1fa6a6dd7303f65a379fcb3733e063d20d1bbc2c76e6fa1 | +| github.com/mattn/go-localereader | v0.0.1 | MIT (README declaration) | README.md | 0c5c52517f13becd7a1e1234f1e8a5ba370f5a5078cee9ac058c2d534e267fcb | +| github.com/mattn/go-runewidth | v0.0.16 | MIT | LICENSE | 88a2379b3ca34bf5c57127aff9dcb802bbb60ece0805cdbda65b3bd115f971d9 | +| github.com/muesli/ansi | v0.0.0-20230316100256-276c6243b2f6 | MIT | LICENSE | 43f65e00e8919fa43c5f45bb2ae578b6d3ee823eb99aa14c85c9ecd06e0350b5 | +| github.com/muesli/cancelreader | v0.2.2 | MIT | LICENSE | 8c139dbbc83b4fb055b4100b6b9402f7de3dd6bbca05015bdfcc4b392c8cfd5f | +| github.com/muesli/termenv | v0.16.0 | MIT | LICENSE | 233e8cf9a812999ccee4d3edb270b0f11ab3539658e680bcb873604915ca5038 | +| github.com/rivo/uniseg | v0.4.7 | MIT | LICENSE.txt | a59885f5f0f3b3c07cf9444db5fe399b6f0791eac82055f7b85cc65500551039 | +| github.com/santhosh-tekuri/jsonschema/v5 | v5.3.1 | Apache-2.0 | LICENSE | c8858a5a76440bbca484e134cf7df46385d090dd18b2c58e650f939258802e5b | +| github.com/spf13/cobra | v1.10.2 | Apache-2.0 | LICENSE.txt | 5e3400b93bbb099e83e52bab885e7441750673c21f97988ca3f1240639b63283 | +| github.com/spf13/pflag | v1.0.9 | BSD-3-Clause | LICENSE | b8514c577c1c4b46cee454d5a882b15fa411e72c5bd7f801f241591789fce61a | +| github.com/xo/terminfo | v0.0.0-20220910002029-abceb7e1c41e | MIT | LICENSE | 1f5f51151ba96f0492ce288bd007242f924fdad9f04002e175e951644340a891 | +| golang.org/x/exp | v0.0.0-20220909182711-5c715a9e8561 | BSD-3-Clause | LICENSE | 2d36597f7117c38b006835ae7f537487207d8ec407aa9d9980794b2030cbc067 | +| golang.org/x/sys | v0.50.0 | BSD-3-Clause | LICENSE | 911f8f5782931320f5b8d1160a76365b83aea6447ee6c04fa6d5591467db9dad | +| golang.org/x/term | v0.48.0 | BSD-3-Clause | LICENSE | 911f8f5782931320f5b8d1160a76365b83aea6447ee6c04fa6d5591467db9dad | +| golang.org/x/text | v0.44.0 | BSD-3-Clause | LICENSE | 911f8f5782931320f5b8d1160a76365b83aea6447ee6c04fa6d5591467db9dad | + +The README screenshot displays the implemented Windows PTY output through a +temporary xterm.js 6.0.0 preview. That QA dependency and the preview server live in +ignored `bin/`; neither is a runtime dependency, shipped feature or CI requirement. + +Native Windows tests/race and the bounded renderer lifecycle tests cover terminal +integration. Native Windows/Ubuntu/macOS GitHub CI passed tests, race and builds; +compilation, model tests and executed CI are recorded separately in [M4 validation](m4-validation.md). diff --git a/docs/testing/m4-native-terminal.json b/docs/testing/m4-native-terminal.json new file mode 100644 index 0000000..0c6f576 --- /dev/null +++ b/docs/testing/m4-native-terminal.json @@ -0,0 +1,89 @@ +{ + "recordedDateLocal": "2026-11-19", + "timezone": "Asia/Calcutta", + "platform": "windows/amd64", + "probe": "tests/terminal/main_windows.go", + "fixture": "docs/testing/fixtures/m4-viewer", + "cases": [ + { + "name": "quit", + "exitCode": 2, + "before": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "after": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "stateRestored": true, + "frameFailureInjected": false, + "status": "native console" + }, + { + "name": "interrupt", + "exitCode": 3, + "before": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "after": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "stateRestored": true, + "frameFailureInjected": false, + "status": "native console" + }, + { + "name": "output-failure", + "exitCode": 3, + "before": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "after": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "stateRestored": true, + "frameFailureInjected": true, + "status": "native console" + }, + { + "name": "resize-and-quit", + "exitCode": 2, + "before": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "after": { + "inputMode": 503, + "outputMode": 7, + "cursorVisible": true + }, + "stateRestored": true, + "frameFailureInjected": false, + "status": "native console", + "resizeAttempts": [ + { + "width": 40, + "height": 18, + "applied": true + }, + { + "width": 96, + "height": 28, + "applied": true + } + ] + } + ] +} diff --git a/docs/testing/m4-validation.md b/docs/testing/m4-validation.md new file mode 100644 index 0000000..6e09e44 --- /dev/null +++ b/docs/testing/m4-validation.md @@ -0,0 +1,220 @@ +# EnvDoctor M4 validation — 2026-11-19 + +ENV-024–ENV-026 are implemented and verified on `feat/m4-implementation`. +Implementation commit: `c532dc33bd16271fb19298cf600a5dd6b05cceb9`. +The branch includes published preparation `2be420a`, based on merged M3 `6cf3581`. +[ADR 0006](../adr/0006-m4-reports-and-bundles.md) was accepted and committed at `32da338` +before dependent implementation. Accepted ADRs and all 172 original golden fixtures are unchanged. + +**The feature branch is published and native Windows/Ubuntu/macOS M4 GitHub CI passed.** +[PR #4](https://github.com/Shellghostt/EnvDoctor/pull/4) is open for review. This record covers local and +CI verification and their limits; signed releases, a main merge and native macOS interactive observation +remain separate work. +Recorded dates use Asia/Calcutta; console captures also contain the corresponding 2026-11-18 UTC time. + +## Environment and canonical gates + +Windows 11 amd64, AMD Ryzen 5 5600H, Go 1.26.11, GNU Make 4.4.1 and golangci-lint 2.15.0. +Every local Go command pins `GOTOOLCHAIN=go1.26.11`. Native race uses the existing MSYS2 UCRT64 +x64 GCC/G++ beneath `%LOCALAPPDATA%/Programs/MSYS2-Go-x64/msys64/ucrt64/bin`. +[Dependency review](m4-dependencies.md) records exact selected modules and license evidence. + +| Check | Executed result | +|---|---| +| `make test` | Passed all packages, including 172 original built-CLI goldens, M4 formats/CLI, sharing, schemas, archive abuse and atomic-output cases | +| Native Windows `make race` | Passed all packages with the x64 compiler; no compiler-limitation skip | +| `make lint` | Passed gofmt, vet and golangci-lint: **0 issues** | +| `make build` | Passed; Windows amd64, CGO disabled, 5,479,936 bytes | +| `make check-docs` | Passed; all twelve generated rule pages remain unchanged | +| Linux `make race` | Passed in `golang:1.26.11`, using a read-only source mount; built-CLI integration package: 22.465 s | +| Linux docs/static build | `make check-docs` and CGO-disabled static build passed; build metadata identifies linux/amd64 and implementation `c532dc3` | +| Four remaining static cross-builds | darwin/amd64, darwin/arm64, windows/arm64 and linux/arm64 passed with CGO disabled | +| `govulncheck ./...` | **No vulnerabilities found** | +| `go mod verify` | All modules verified | +| Bounded `FuzzM4BundleReader` run | Passed: 15 s fuzz budget, two workers, 3,158 executions, one new interesting input; no failure corpus | +| Documentation and local review | `make check-docs`, local Markdown file links, all 40 named M4 test/fuzz/benchmark functions and `git diff --check` passed; golden/registry/ADR files preserved | +| Native three-OS M4 GitHub CI | **Passed**, all five jobs in both PR and push workflows on `5eff1dd`; executed evidence below | + +Static build sizes were 5,173,410 bytes for linux/amd64; 4,915,362 for linux/arm64; +5,245,904 for darwin/amd64; 4,989,714 for darwin/arm64; and 5,106,688 for windows/arm64. +These compile checks cover all six targets and stay below the 25 MB target. They are not signed release +or native macOS execution evidence. GoReleaser packaging was not rerun for M4. + +## GitHub CI verification + +Publication was authorized on 2026-11-19. The pushed implementation and local validation record, +`5eff1dd4883ddbed72fc059d85fca9493493e646`, passed both workflows: + +| Workflow | Run | Result | +|---|---|---| +| Pull request | [44421272870](https://github.com/Shellghostt/EnvDoctor/actions/runs/44421272870) | **5/5 jobs passed** | +| Feature push | [44421244163](https://github.com/Shellghostt/EnvDoctor/actions/runs/44421244163) | **5/5 jobs passed** | + +Each native Windows/Ubuntu/macOS job executed formatting, vet, generated-document checks, +`go test -race -count=1 -timeout=120s ./...` and `go build ./...`. Lint and `govulncheck` also passed. +These runs execute the M4 source on native runners; local PTY observations and performance measurements +remain distinct evidence. PR #4 exposes checks for the current branch head after documentation updates. + +## Acceptance mapping + +| M4 criterion | Named executed evidence | +|---|---| +| TTY, redirects, `TERM=dumb`, empty/present `NO_COLOR`, explicit plain/format precedence and uncontaminated structured output | `TestM4FormatPolicy`, `TestM4CLIFormatAndTerminalMatrix`, `TestM4CLIInvalidFormatsBeforeCollection`, `TestM4CLIStructuredOutputErrors`, `TestM4EveryFormatPropagatesShortWrites` | +| Narrow/resize/scroll, interruption, output failure and terminal/timer cleanup | `TestM4ViewerNarrowResizeScrollAndInterrupt`, `TestM4InteractiveQuitCancellationAndOutputCleanup`, `TestM4ResizePollerStopsAndJoinsOnCancellation`, `TestM4RepeatedViewerLifecycleReleasesResources`; native console probe below | +| Markdown severity, order, limitations, errors, omissions and hostile content | Reviewed `internal/report/testdata/markdown-m4.golden`; `TestM4MarkdownGoldenAndEscaping`, `TestM4MarkdownBoundsAndBidiControls`, `TestM4TextControlsAndMarkdownWriteFailure` | +| Exclusive compressed-size bound, streaming/member/parse limits and explicit failure | `TestM4ArchiveExactCompressedBoundary`, `TestM4ArchiveBufferCannotBypassSizeGuard`, `TestM4JSONIndependentWorkBounds`, `TestM4DeclaredAndHiddenDecompressionBombAreBounded` | +| Sharing privacy across every member and diagnostics; unsupported fields/values remain unavailable | `TestM4BundleSharingAllowlistAndPresence`, `TestM4UnsupportedSharingValuesAreInconclusive`, source-model coverage guard, offline/error privacy and structured-log CLI cases | +| Corrupt, duplicate, traversal, symlink, unsupported and malformed archives | `TestM4ArchiveHeaderAbuse`, `TestM4ArchiveNamesAndTypes`, `TestM4BundleStrictJSONAndMetadata`, `TestM4DuplicateEntryBelowEntryLimit`, `TestM4StoreDeflateAndCompressedTrailingGarbage`, bounded fuzz run | +| Windows/Linux/macOS evidence survives another receiving OS | `TestM4All172GoldenCapturesRoundTripAcrossOS`, `TestM4EnvironmentAliasUsesCapturedOSAndExactPaths`, executed on Windows and Linux; actual Windows bundle read in network-disabled Linux below | +| Offline collector/home/environment bypass, provenance, flags, exits and saved-verdict independence | `TestM4CLIOfflineEvidenceAndProvenance`, `TestM4CLIOfflineAllExitCodes`, `TestM4CLIConflictsAndReadFailuresAreOffline`, `TestM4SavedVerdictsAreReferenceOnly` | +| Schema compatibility and required structure validation | `TestM4BundleMembersMatchCommittedSchemas`, `TestM4OptionalEmptyAliasesMatchSchemaAndReader`, original diagnose/snapshot schema and built-CLI tests | +| Atomic output failure, cancellation, locked destinations and private staging | `TestM4PublicationEveryStagePreservesDestination`, `TestM4AtomicPublicationAndCancellation`, `TestM4SymlinkDestinationAndInputAreNotFollowed`, `TestM4WindowsAtomicReplaceLockedDestination`, `TestM4WindowsPublicationHasPrivateProtectedACL`, `TestM4CLIBundleStatusFailurePreservesDestination` | +| M1–M3 collectors, rules, JSON, registry, goldens and opt-in probe contracts | Full canonical test/race suites; original 172 goldens and twelve rule pages have no diff | +| Pinned/audited terminal dependencies and local/native CI gates | [Dependency review](m4-dependencies.md), canonical gate and GitHub CI tables above | +| Real README screenshot | [m4-terminal.jpg](../images/m4-terminal.jpg), from actual CLI Windows PTY output using committed synthetic identities/paths | +| Measured lifecycle/time/memory limits and honest remaining work | Resource measurements, native evidence and interactive/performance/release limits below | + +The runtime reader uses exact typed JSON grammar preflight and semantic validation. Schema tests compile +the committed schemas with network loading forbidden. Ordinary report-v1 properties/values are unchanged; +the only compatible addition is optional bundle-source metadata, absent from ordinary live/fixture runs. +Empty optional aliases are accepted by both reader and schema; raw paths and aliases outside 1–4,096 fail. + +## Independent archive and evidence bounds + +| Resource | Enforced contract | +|---|---| +| Complete compressed ZIP, including directory | Fewer than 2,000,000 bytes; 1,999,999 accepted and 2,000,000 rejected | +| Total streamed uncompressed member bytes | At most 8 MiB | +| Entries | At most eight; five literal required names and optional `diagnostics.json` | +| Manifest / versions | 64 KiB each | +| Snapshot / Markdown | 2 MiB each | +| JSON report / diagnostics | 3 MiB / 512 KiB | +| JSON grammar/work | One document; exact tags; required fields; no unknown/duplicate/case aliases; depth 32, 200,000 tokens | +| JSON scalar/array work | Strings at most 4,096 UTF-8 bytes; arrays at most 4,096 elements; numeric tokens at most 32 bytes | +| PATH / runtimes | 1,024 / 128 | +| Listeners / requested ports | 1,024 each | +| Dependencies / proxies / errors / omissions | 32 / 3 / 32 / 256 | + +The reader checks the entire central/local layout before streaming. It accepts regular Store/Deflate +entries, verifies CRC/SHA-256 and cross-member metadata, and rejects unsupported versions/flags, encryption, +ZIP64/disks, extra/comments, overlaps, prefixes/trailing bytes, duplicate/nonliteral names and symlinks. +Deflate byte reading detects trailing compressed garbage. No member is extracted or executed. +Digests detect corruption, not authorship or authentic machine state. + +Publication protects the empty owned stage before payload writes, syncs/closes/revalidates it, then +performs same-directory replacement. Injected protect/write/short-write/sync/close/read/validation/rename +and cancellation failures preserve the previous destination and remove the owned temporary file. +A status-stream failure is detected before publication. Atomic namespace replacement assumes a supporting +local filesystem; network filesystem behavior and power-loss durability are not promised. + +## Native terminal evidence + +The committed manual harness [tests/terminal/main_windows.go](../../tests/terminal/main_windows.go) +calls the production CLI with real console handles. It can inject one transient frame failure and +exercise two owned viewport sizes. [m4-native-terminal.json](m4-native-terminal.json) records the results. + +| Scenario | Result | Console modes/cursor after exit | +|---|---|---| +| `q` | Diagnosis exit 2 and plain report | Restored | +| Ctrl+C | Exit 3 and plain report | Restored | +| Injected frame-write failure | Exit 3; injection confirmed; plain fallback attempted | Restored | +| Resize 40×18 → 96×28, then `q` | Both native viewport changes applied; report rewrapped; exit 2 | Restored | + +Before/after input mode was 503, output mode 7, and the cursor was visible. Bubble Tea v1.3.10 has no +Windows resize listener, so the adapter owns a 200 ms window-size poller and cancels/joins it on every +return. Model tests also cover 1×1, 2×2, Unicode widths, scroll boundaries and oversized dimensions. + +The README image displays actual live Windows PTY ANSI output through a temporary xterm.js 6.0.0 +preview. It is not a physical desktop screenshot. The preview server and PTY were stopped after capture; +neither that server nor Node is a product runtime dependency. Native restoration/resize evidence is +independent of the browser display. + +## Real Windows capture and offline cross-OS smoke + +The final built Windows CLI published a 4,524-byte bundle with structured diagnostics. Its original +diagnosis exit was 1 for existing PATH warnings. Windows and network-disabled Linux offline diagnosis +both returned 3 with identical findings/source metadata: one retained finding and ten explicit omissions. +The captured shell-name value was outside the sharing enum and became a fixed unavailable collector +notice. That is intentional degradation; removing unsupported evidence does not establish health. + +The Linux receiving process preserved Windows capture OS/time, current evaluator provenance and opaque +aliases. `--network none` blocked remote network work; the CLI used no collectors. Automated offline +tests also inject panic collectors/home/environment accessors and cover every diagnosis exit and flag +conflict. The saved JSON verdict/Markdown tampering tests confirm evaluation uses current builtin rules. + +## Lifecycle, allocations and local performance + +With `GOMAXPROCS=4`, 100 synthetic bundle build/read/cancel cycles took **303.2657 ms**; the archive was +7,342 bytes. Goroutines stayed **2 → 2**; retained heap after GC changed **1,797,600 → 887,320 bytes**. +Fifty synthetic viewer quit/cancel cycles took **533.4757 ms**, with goroutines **2 → 2** and retained +heap **304,424 → 285,560 bytes**. The resize-poller cancellation/join test also passed. + +Twenty-five hidden decompression bombs were rejected in **1.0419 ms**, allocating **2,140,424 bytes +total**. A 100-iteration bundle build/read benchmark reported **3,002,830 ns/op**, **1,535,852 B/op** +and **11,419 allocs/op**. The sampled retained profile contained runtime and test-schema initialization +allocations; it is a sampled workload result, not proof that every possible input is leak-free. + +Ten read-only full Windows diagnoses on `c532dc3` completed in milliseconds: +561.36, 236.06, 240.98, 245.00, 247.62, 247.96, 249.09, 248.41, 242.53 and 245.23. +All returned exit 1 with nine collectors and twelve rules. Windows `GetProcessMemoryInfo` counters, +including successful final reads after exit, reported peak working sets from **13,213,696 to 14,782,464 +bytes**. These local warm runs stayed below the 3 s/100 MB targets. They are not cold-host, native macOS +or universal process-memory results. Machine PATH/configuration was not changed. + +Profiles, actual local captures, binaries and raw command logs stay beneath ignored `bin/`. Shared +fixtures use synthetic data. The existing Docker data/workloads were preserved; owned Linux test/smoke +containers used `--rm` and left no running M4 containers. + +## Reproduce + +```powershell +$env:GOTOOLCHAIN='go1.26.11' +$raceCompilerDir=Join-Path $env:LOCALAPPDATA 'Programs/MSYS2-Go-x64/msys64/ucrt64/bin' +$env:CC=Join-Path $raceCompilerDir 'gcc.exe' +$env:CXX=Join-Path $raceCompilerDir 'g++.exe' +$env:PATH="$raceCompilerDir;$env:PATH" +$env:CGO_ENABLED='1' +make test +make race +make lint +make build +make check-docs +govulncheck ./... +go mod verify +git diff --check +``` + +The Linux race/docs/static gates ran in `golang:1.26.11`, with `GOTOOLCHAIN=local`, `GOMAXPROCS=4`, a +read-only repository mount at `/src`, and an owned ignored output directory mounted at `/qaout`: + +```sh +make race +make check-docs +CGO_ENABLED=0 go build -trimpath -ldflags '-s -w -X main.version=m4-validation' -o /qaout/envdoctor ./cmd/envdoctor +go version -m /qaout/envdoctor +``` + +The resulting binary was run in a second `--network none` container against the actual Windows bundle. +For bounded fuzzing and allocation measurements: + +```powershell +$env:GOTOOLCHAIN='go1.26.11' +$env:GOMAXPROCS='4' +go test ./internal/bundle -run '^$' -fuzz FuzzM4BundleReader -fuzztime=15s -parallel=2 +go test -count=1 -run 'TestM4RepeatedBundleLifecycleReleasesResources|TestM4DeclaredAndHiddenDecompressionBombAreBounded' -v ./internal/bundle +go test -count=1 -run 'TestM4RepeatedViewerLifecycleReleasesResources|TestM4ResizePollerStopsAndJoinsOnCancellation' -v ./internal/report +go test -run '^$' -bench '^BenchmarkM4BundleBuildRead$' -benchtime=100x -benchmem -memprofile bin/m4-bundle-mem.pprof -o bin/m4-bundle-profile.test.exe ./internal/bundle +go tool pprof -top -inuse_space bin/m4-bundle-mem.pprof +``` + +Native console reproduction: build `./tests/terminal` into `bin/m4-terminal-probe.exe`, run it with +`--record`, optional `--resize`/`--fail-frame`, then `diagnose --fixture-dir docs/testing/fixtures/m4-viewer`. +Use a capable console with `TERM=xterm-256color` and `NO_COLOR` absent. The harness only changes its +owned viewport and records modes/cursor; production diagnosis has no testing-only flags. + +## Review and later milestones + +The M4 branch is published in PR #4 with passing native Windows/Ubuntu/macOS tests, race and builds, +lint and vulnerability scanning. Review and merging remain separate decisions. Native macOS +interactive observation, cold/clean-host performance, packaging and signed releases remain later work. +Rules ENV013–ENV018 and fix/apply/undo remain later milestones. diff --git a/go.mod b/go.mod index 47b3336..bf807a0 100644 --- a/go.mod +++ b/go.mod @@ -5,12 +5,31 @@ go 1.26.0 toolchain go1.26.11 require ( + github.com/charmbracelet/bubbletea v1.3.10 + github.com/charmbracelet/x/ansi v0.10.1 github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 github.com/spf13/cobra v1.10.2 - golang.org/x/sys v0.48.0 + golang.org/x/sys v0.50.0 + golang.org/x/term v0.48.0 ) require ( + github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect + github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect + github.com/charmbracelet/lipgloss v1.1.0 // indirect + github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd // indirect + github.com/charmbracelet/x/term v0.2.1 // indirect + github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/lucasb-eyer/go-colorful v1.2.0 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/mattn/go-localereader v0.0.1 // indirect + github.com/mattn/go-runewidth v0.0.16 // indirect + github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect + github.com/muesli/cancelreader v0.2.2 // indirect + github.com/muesli/termenv v0.16.0 // indirect + github.com/rivo/uniseg v0.4.7 // indirect github.com/spf13/pflag v1.0.9 // indirect + github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect + golang.org/x/text v0.44.0 // indirect ) diff --git a/go.sum b/go.sum index 4830213..34bd26b 100644 --- a/go.sum +++ b/go.sum @@ -1,6 +1,39 @@ +github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k= +github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8= +github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw= +github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4= +github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc h1:4pZI35227imm7yK2bGPcfpFEmuY1gc2YSTShr4iJBfs= +github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc/go.mod h1:X4/0JoqgTIPSFcRA/P6INZzIuyqdFY5rm8tb41s9okk= +github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY= +github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30= +github.com/charmbracelet/x/ansi v0.10.1 h1:rL3Koar5XvX0pHGfovN03f5cxLbCF2YvLeyz7D2jVDQ= +github.com/charmbracelet/x/ansi v0.10.1/go.mod h1:3RQDQ6lDnROptfpWuUVIUG64bD2g2BgntdxH0Ya5TeE= +github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd h1:vy0GVL4jeHEwG5YOXDmi86oYw2yuYUGqz6a8sLwg0X8= +github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd/go.mod h1:xe0nKWGd3eJgtqZRaN9RjMtK7xUYchjzPr7q6kcvCCs= +github.com/charmbracelet/x/term v0.2.1 h1:AQeHeLZ1OqSXhrAWpYUtZyX1T3zVxfpZuEQMIQaGIAQ= +github.com/charmbracelet/x/term v0.2.1/go.mod h1:oQ4enTYFV7QN4m0i9mzHrViD7TQKvNEEkHUMCmsxdUg= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4= +github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY= +github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4= +github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88= +github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= +github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= +github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI= +github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo= +github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA= +github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo= +github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc= +github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk= +github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= +github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= +github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 h1:lZUw3E0/J3roVtGQ+SCrUrg3ON6NgVqpn3+iol9aGu4= github.com/santhosh-tekuri/jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY= @@ -8,7 +41,17 @@ github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no= +github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= -golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo= -golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= +golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561 h1:MDc5xs78ZrZr3HMQugiXOAkSZtfTpbJLDr/lwfgO53E= +golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE= +golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.50.0 h1:dkKZEFTEKaLr1sjhs0WiKqoi8Jj5nHbS3Njmx1UoMJs= +golang.org/x/sys v0.50.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= +golang.org/x/term v0.48.0 h1:G6UU1eiSB8nl3DzeUGt7Mg+2SctI8tPS6Z5/mBt6cOk= +golang.org/x/term v0.48.0/go.mod h1:BRDLHCZr2SUjIeBHHpRyQeC9xzSAVbzY2rvKgafnHf4= +golang.org/x/text v0.44.0 h1:YItdp/mQNGnsmrgpcdA5yQ3bnRxCfo/Tj/iByGW9GGo= +golang.org/x/text v0.44.0/go.mod h1:S4FR1waQ9LbjQTls97SyRjDaUUVkaM+cXwwuG/G+MwY= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= diff --git a/internal/bundle/archive.go b/internal/bundle/archive.go new file mode 100644 index 0000000..8a60b88 --- /dev/null +++ b/internal/bundle/archive.go @@ -0,0 +1,385 @@ +package bundle + +import ( + "archive/zip" + "bytes" + "compress/flate" + "context" + "crypto/sha256" + "encoding/binary" + "encoding/hex" + "encoding/json" + "hash/crc32" + "io" + "sort" + "unicode/utf8" + + "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" +) + +type archiveBuffer struct { + ctx context.Context + buffer bytes.Buffer +} + +func (b *archiveBuffer) Len() int { return b.buffer.Len() } +func (b *archiveBuffer) Bytes() []byte { return b.buffer.Bytes() } + +func (b *archiveBuffer) Write(data []byte) (int, error) { + if err := b.ctx.Err(); err != nil { + return 0, err + } + if len(data) >= MaxArchiveBytes-b.Len() { + return 0, ErrBounds + } + return b.buffer.Write(data) +} + +func marshalMember(ctx context.Context, name string, value any) ([]byte, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + data, err := json.Marshal(value) + if err != nil { + return nil, ErrEvidence + } + if len(data) > memberLimits[name] { + return nil, ErrBounds + } + return data, nil +} + +// Build recomputes all report members from the sharing model with the builtin +// engine. It never zips the original document, findings, errors or log text. +func Build(ctx context.Context, original report.Document, opts Options) ([]byte, error) { + snapshot, source, timings, diags, err := export(ctx, original) + if err != nil { + return nil, err + } + if err := validateSnapshot(snapshot); err != nil { + return nil, err + } + evidence := engineSnapshot(snapshot) + findings, err := diagnostic.DefaultEngine().Evaluate(evidence, diagnostic.Filter{}) + if err != nil { + return nil, ErrEvidence + } + doc := report.NewDocument(source.CaptureTool.Version, source.CapturedAt, evidence, findings, timings, len(diagnostic.BuiltinMetadata())) + doc.Source = &source + all := map[string][]byte{} + for _, member := range []struct { + name string + value any + }{{"snapshot.json", snapshot}, {"report.json", doc}, {"versions.json", makeVersions(snapshot, source.CaptureTool)}} { + all[member.name], err = marshalMember(ctx, member.name, member.value) + if err != nil { + return nil, err + } + } + var markdown bytes.Buffer + if err := report.WriteMarkdown(&markdown, doc); err != nil { + return nil, ErrBounds + } + all["report.md"] = markdown.Bytes() + if opts.IncludeLogs { + all["diagnostics.json"], err = marshalMember(ctx, "diagnostics.json", diags) + if err != nil { + return nil, err + } + } + manifest := Manifest{BundleVersion: 1, SnapshotVersion: 1, ReportVersion: 1, RedactionPolicy: Policy, Tool: source.CaptureTool, CapturedAt: source.CapturedAt, CaptureExitCode: source.CaptureExitCode, Omissions: source.Omissions, Members: []Member{}} + names := []string{"manifest.json", "snapshot.json", "report.json", "report.md", "versions.json"} + if opts.IncludeLogs { + names = append(names, "diagnostics.json") + } + total := 0 + for _, name := range names[1:] { + body := all[name] + digest := sha256.Sum256(body) + manifest.Members = append(manifest.Members, Member{Name: name, Size: len(body), SHA256: hex.EncodeToString(digest[:])}) + total += len(body) + } + all["manifest.json"], err = marshalMember(ctx, "manifest.json", manifest) + if err != nil { + return nil, err + } + if total+len(all["manifest.json"]) > MaxUncompressedBytes { + return nil, ErrBounds + } + out := archiveBuffer{ctx: ctx} + w := zip.NewWriter(&out) + for _, name := range names { + header := &zip.FileHeader{Name: name, Method: zip.Deflate} + // No extended timestamp/extra/comment records or platform paths. + header.SetMode(0600) + stream, err := w.CreateHeader(header) + if err != nil { + return nil, err + } + if _, err := stream.Write(all[name]); err != nil { + return nil, err + } + } + if err := w.Close(); err != nil { + return nil, err + } + data := out.Bytes() + if _, err := Read(ctx, data); err != nil { + return nil, err + } + return data, nil +} + +type zipEntry struct { + name string + offset, compressed, uncompressed int + crc uint32 + flags, method uint16 + version, modifiedTime, modifiedDate uint16 +} + +// Preflight the entire directory and every local record BEFORE archive/zip can +// allocate its file graph. Offsets must describe one exact nonoverlapping ZIP, +// without a prefix, trailing records, extras, ZIP64 or another disk. +func preflight(data []byte) ([]zipEntry, error) { + if len(data) >= MaxArchiveBytes { + return nil, ErrBounds + } + if len(data) < 22 { + return nil, ErrInvalid + } + eocd := len(data) - 22 + end := data[eocd:] + u16 := binary.LittleEndian.Uint16 + u32 := binary.LittleEndian.Uint32 + if u32(end) != 0x06054b50 || u16(end[4:]) != 0 || u16(end[6:]) != 0 || u16(end[20:]) != 0 { + return nil, ErrInvalid + } + count := int(u16(end[10:])) + if count > MaxEntries { + return nil, ErrBounds + } + if count < 5 || count != int(u16(end[8:])) { + return nil, ErrInvalid + } + dirSize, dirStart := uint64(u32(end[12:])), uint64(u32(end[16:])) + if dirStart+dirSize != uint64(eocd) || dirStart > uint64(eocd) { + return nil, ErrInvalid + } + position := int(dirStart) + entries := make([]zipEntry, 0, count) + seen := map[string]bool{} + total := 0 + for i := 0; i < count; i++ { + if position > eocd-46 { + return nil, ErrInvalid + } + h := data[position : position+46] + if u32(h) != 0x02014b50 || u16(h[6:]) != 10 && u16(h[6:]) != 20 || u16(h[34:]) != 0 { + return nil, ErrInvalid + } + flags, method := u16(h[8:]), u16(h[10:]) + if flags & ^uint16(0x0808) != 0 || method != zip.Store && method != zip.Deflate { + return nil, ErrInvalid + } + if method == zip.Deflate && u16(h[6:]) != 20 { + return nil, ErrInvalid + } + nameSize, extraSize, commentSize := int(u16(h[28:])), int(u16(h[30:])), int(u16(h[32:])) + if extraSize != 0 || commentSize != 0 || nameSize < 1 || nameSize > 16 || position+46+nameSize > eocd { + return nil, ErrInvalid + } + name := string(data[position+46 : position+46+nameSize]) + limit, ok := memberLimits[name] + if !ok || seen[name] { + return nil, ErrInvalid + } + seen[name] = true + creator := u16(h[4:]) + if creator>>8 != 0 && creator>>8 != 3 { + return nil, ErrInvalid + } + mode := (&zip.FileHeader{Name: name, CreatorVersion: creator, ExternalAttrs: u32(h[38:])}).Mode() + if !mode.IsRegular() { + return nil, ErrInvalid + } + attrs := u32(h[38:]) + if creator>>8 == 0 && attrs>>16 != 0 || creator>>8 == 3 && attrs>>28 != 0 && attrs>>28 != 8 || attrs&0x18 != 0 { + return nil, ErrInvalid + } + compressed, uncompressed, offset := uint64(u32(h[20:])), uint64(u32(h[24:])), uint64(u32(h[42:])) + if uncompressed > uint64(limit) || compressed >= MaxArchiveBytes { + return nil, ErrBounds + } + if method == zip.Store && compressed != uncompressed { + return nil, ErrIntegrity + } + if offset > dirStart || offset+30+uint64(nameSize)+compressed > dirStart { + return nil, ErrInvalid + } + total += int(uncompressed) + if total > MaxUncompressedBytes { + return nil, ErrBounds + } + entries = append(entries, zipEntry{name: name, offset: int(offset), compressed: int(compressed), uncompressed: int(uncompressed), crc: u32(h[16:]), flags: flags, method: method, version: u16(h[6:]), modifiedTime: u16(h[12:]), modifiedDate: u16(h[14:])}) + position += 46 + nameSize + } + if position != eocd { + return nil, ErrInvalid + } + for _, name := range requiredMembers { + if !seen[name] { + return nil, ErrInvalid + } + } + sort.Slice(entries, func(i, j int) bool { return entries[i].offset < entries[j].offset }) + position = 0 + for _, entry := range entries { + if entry.offset != position || position > int(dirStart)-30 { + return nil, ErrInvalid + } + h := data[position : position+30] + if u32(h) != 0x04034b50 || u16(h[4:]) != entry.version || u16(h[6:]) != entry.flags || u16(h[8:]) != entry.method || u16(h[10:]) != entry.modifiedTime || u16(h[12:]) != entry.modifiedDate || int(u16(h[26:])) != len(entry.name) || u16(h[28:]) != 0 { + return nil, ErrInvalid + } + if string(data[position+30:position+30+len(entry.name)]) != entry.name { + return nil, ErrInvalid + } + if entry.flags&8 == 0 { + if u32(h[14:]) != entry.crc || u32(h[18:]) != uint32(entry.compressed) || u32(h[22:]) != uint32(entry.uncompressed) { + return nil, ErrIntegrity + } + } else { + if u32(h[14:]) != 0 && u32(h[14:]) != entry.crc || u32(h[18:]) != 0 && u32(h[18:]) != uint32(entry.compressed) || u32(h[22:]) != 0 && u32(h[22:]) != uint32(entry.uncompressed) { + return nil, ErrIntegrity + } + } + position += 30 + len(entry.name) + entry.compressed + if entry.flags&8 != 0 { + if position > int(dirStart)-12 { + return nil, ErrInvalid + } + if u32(data[position:]) == 0x08074b50 { + position += 4 + } + if position > int(dirStart)-12 { + return nil, ErrInvalid + } + if u32(data[position:]) != entry.crc || u32(data[position+4:]) != uint32(entry.compressed) || u32(data[position+8:]) != uint32(entry.uncompressed) { + return nil, ErrIntegrity + } + position += 12 + } + } + if position != int(dirStart) { + return nil, ErrInvalid + } + return entries, nil +} + +type contextReader struct { + ctx context.Context + input io.Reader +} + +// Implement flate.Reader's byte method so the inflater cannot read ahead and +// hide garbage after a complete Deflate stream inside its compressed extent. +type compressedReader struct { + ctx context.Context + reader *bytes.Reader +} + +func (r compressedReader) Read(p []byte) (int, error) { + if err := r.ctx.Err(); err != nil { + return 0, err + } + return r.reader.Read(p) +} +func (r compressedReader) ReadByte() (byte, error) { + if err := r.ctx.Err(); err != nil { + return 0, err + } + return r.reader.ReadByte() +} + +func (r contextReader) Read(data []byte) (int, error) { + if err := r.ctx.Err(); err != nil { + return 0, err + } + return r.input.Read(data) +} + +// Read validates all archive/reference members but returns only typed evidence +// and provenance. No saved verdict, Markdown, command or URL is returned. +func Read(ctx context.Context, data []byte) (Capture, error) { + if err := ctx.Err(); err != nil { + return Capture{}, err + } + entries, err := preflight(data) + if err != nil { + return Capture{}, err + } + all := make(map[string][]byte, len(entries)) + total := 0 + for _, entry := range entries { + if err := ctx.Err(); err != nil { + return Capture{}, err + } + start := entry.offset + 30 + len(entry.name) + compressed := compressedReader{ctx: ctx, reader: bytes.NewReader(data[start : start+entry.compressed])} + stream := io.NopCloser(io.Reader(compressed)) + if entry.method == zip.Deflate { + stream = flate.NewReader(compressed) + } + body, readErr := io.ReadAll(io.LimitReader(stream, int64(entry.uncompressed)+1)) + closeErr := stream.Close() + if err := ctx.Err(); err != nil { + return Capture{}, err + } + if len(body) > memberLimits[entry.name] || total+len(body) > MaxUncompressedBytes { + return Capture{}, ErrBounds + } + if readErr != nil || closeErr != nil || len(body) != entry.uncompressed || compressed.reader.Len() != 0 || crc32.ChecksumIEEE(body) != entry.crc { + return Capture{}, ErrIntegrity + } + all[entry.name] = body + total += len(body) + } + var manifest Manifest + if !utf8.Valid(all["report.md"]) { + return Capture{}, ErrInvalid + } + if err := decodeJSON(ctx, all["manifest.json"], &manifest); err != nil { + return Capture{}, err + } + if err := validateManifest(manifest, all); err != nil { + return Capture{}, err + } + var snapshot SharingSnapshot + if err := decodeJSON(ctx, all["snapshot.json"], &snapshot); err != nil { + return Capture{}, err + } + if err := validateSnapshot(snapshot); err != nil { + return Capture{}, err + } + var doc report.Document + if err := decodeJSON(ctx, all["report.json"], &doc); err != nil { + return Capture{}, err + } + var versions Versions + if err := decodeJSON(ctx, all["versions.json"], &versions); err != nil { + return Capture{}, err + } + var diags *Diagnostics + if data, ok := all["diagnostics.json"]; ok { + diags = &Diagnostics{} + if err := decodeJSON(ctx, data, diags); err != nil { + return Capture{}, err + } + } + if err := validateReference(doc, snapshot, manifest, versions, diags); err != nil { + return Capture{}, err + } + return Capture{Snapshot: engineSnapshot(snapshot), Source: sourceFor(manifest)}, nil +} diff --git a/internal/bundle/archive_test.go b/internal/bundle/archive_test.go new file mode 100644 index 0000000..28624e2 --- /dev/null +++ b/internal/bundle/archive_test.go @@ -0,0 +1,326 @@ +package bundle_test + +import ( + "archive/zip" + "bytes" + "context" + "crypto/sha256" + "encoding/binary" + "encoding/hex" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/shellghostt/envdoctor/internal/bundle" +) + +// These archive cases were written before the archive reader/writer. Repacking +// updates digests so JSON/semantic rejection is tested independently of CRCs. +func repack(t *testing.T, all map[string][]byte, method uint16) []byte { + t.Helper() + var m bundle.Manifest + if err := json.Unmarshal(all["manifest.json"], &m); err != nil { + t.Fatal(err) + } + m.Members = nil + for _, name := range []string{"snapshot.json", "report.json", "report.md", "versions.json", "diagnostics.json"} { + if body, ok := all[name]; ok { + digest := sha256.Sum256(body) + m.Members = append(m.Members, bundle.Member{Name: name, Size: len(body), SHA256: hex.EncodeToString(digest[:])}) + } + } + var err error + all["manifest.json"], err = json.Marshal(m) + if err != nil { + t.Fatal(err) + } + var out bytes.Buffer + w := zip.NewWriter(&out) + for _, name := range []string{"manifest.json", "snapshot.json", "report.json", "report.md", "versions.json", "diagnostics.json"} { + body, ok := all[name] + if !ok { + continue + } + h := &zip.FileHeader{Name: name, Method: method} + h.SetMode(0600) + stream, err := w.CreateHeader(h) + if err != nil { + t.Fatal(err) + } + if _, err := stream.Write(body); err != nil { + t.Fatal(err) + } + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + return out.Bytes() +} + +func validArchive(t *testing.T) []byte { + t.Helper() + data, err := bundle.Build(context.Background(), sensitiveDocument(), bundle.Options{}) + if err != nil { + t.Fatal(err) + } + return data +} + +func central(data []byte) int { + return int(binary.LittleEndian.Uint32(data[len(data)-22+16:])) +} + +func TestM4ArchiveHeaderAbuse(t *testing.T) { + base := validArchive(t) + for _, tc := range []struct { + name string + mutate func([]byte) []byte + }{ + {"trailing-data", func(b []byte) []byte { return append(b, 0) }}, + {"prefix-data", func(b []byte) []byte { return append([]byte{0}, b...) }}, + {"multidisk", func(b []byte) []byte { binary.LittleEndian.PutUint16(b[len(b)-22+4:], 1); return b }}, + {"zip64-count", func(b []byte) []byte { binary.LittleEndian.PutUint16(b[len(b)-22+10:], 65535); return b }}, + {"encrypted", func(b []byte) []byte { + binary.LittleEndian.PutUint16(b[6:], 1) + binary.LittleEndian.PutUint16(b[central(b)+8:], 1) + return b + }}, + {"unsupported-method", func(b []byte) []byte { + binary.LittleEndian.PutUint16(b[8:], 99) + binary.LittleEndian.PutUint16(b[central(b)+10:], 99) + return b + }}, + {"local-central-name", func(b []byte) []byte { b[30] = 'X'; return b }}, + {"local-central-flags", func(b []byte) []byte { b[6] ^= 8; return b }}, + {"overlap", func(b []byte) []byte { binary.LittleEndian.PutUint32(b[central(b)+42:], 1); return b }}, + {"central-extra", func(b []byte) []byte { binary.LittleEndian.PutUint16(b[central(b)+30:], 1); return b }}, + {"local-extra", func(b []byte) []byte { binary.LittleEndian.PutUint16(b[28:], 1); return b }}, + {"comment", func(b []byte) []byte { binary.LittleEndian.PutUint16(b[len(b)-22+20:], 1); return b }}, + {"oversized-declared-member", func(b []byte) []byte { binary.LittleEndian.PutUint32(b[central(b)+24:], 9<<20); return b }}, + {"crc", func(b []byte) []byte { b[central(b)+16] ^= 1; return b }}, + {"corrupt-data", func(b []byte) []byte { b[30+len("manifest.json")+3] ^= 0xff; return b }}, + } { + t.Run(tc.name, func(t *testing.T) { + data := tc.mutate(bytes.Clone(base)) + if _, err := bundle.Read(context.Background(), data); err == nil { + t.Fatal("hostile archive accepted") + } + }) + } +} + +func TestM4ArchiveNamesAndTypes(t *testing.T) { + for _, name := range []string{"../report.md", "/report.md", `C:\report.md`, `\\host\report.md`, "a/../report.md", "report.md/", "other.json", "REPORT.MD", "report.md\x00", "a\\report.md"} { + t.Run(name, func(t *testing.T) { + var out bytes.Buffer + w := zip.NewWriter(&out) + h := &zip.FileHeader{Name: name, Method: zip.Store} + h.SetMode(0600) + f, err := w.CreateHeader(h) + if err != nil { + t.Fatal(err) + } + if _, err := f.Write(nil); err != nil { + t.Fatal(err) + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(context.Background(), out.Bytes()); err == nil { + t.Fatal("unsupported name accepted") + } + }) + } + for _, mode := range []os.FileMode{os.ModeSymlink | 0777, os.ModeDir | 0700, os.ModeNamedPipe | 0600} { + var out bytes.Buffer + w := zip.NewWriter(&out) + h := &zip.FileHeader{Name: "manifest.json", Method: zip.Store} + h.SetMode(mode) + f, err := w.CreateHeader(h) + if err != nil { + t.Fatal(err) + } + if _, err := f.Write(nil); err != nil { + t.Fatal(err) + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(context.Background(), out.Bytes()); err == nil { + t.Fatalf("nonregular mode %v accepted", mode) + } + } + var out bytes.Buffer + w := zip.NewWriter(&out) + for i := 0; i < 9; i++ { + f, err := w.Create("manifest.json") + if err != nil { + t.Fatal(err) + } + if _, err := f.Write([]byte("{}")); err != nil { + t.Fatal(err) + } + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(context.Background(), out.Bytes()); err == nil { + t.Fatal("duplicate/excess entries accepted") + } +} + +func TestM4BundleStrictJSONAndMetadata(t *testing.T) { + base := validArchive(t) + for _, tc := range []struct { + name, member string + change func([]byte) []byte + }{ + {"duplicate", "snapshot.json", func(b []byte) []byte { + return bytes.Replace(b, []byte(`"schemaVersion":1`), []byte(`"schemaVersion":1,"schemaVersion":1`), 1) + }}, + {"case-alias", "snapshot.json", func(b []byte) []byte { + return bytes.Replace(b, []byte(`"schemaVersion":1`), []byte(`"schemaVersion":1,"SchemaVersion":1`), 1) + }}, + {"unknown", "snapshot.json", func(b []byte) []byte { return append([]byte(`{"rawSecret":"SECRET-ARCHIVE",`), b[1:]...) }}, + {"missing-required", "snapshot.json", func(b []byte) []byte { return bytes.Replace(b, []byte(`"schemaVersion":1,`), nil, 1) }}, + {"second-document", "snapshot.json", func(b []byte) []byte { return append(b, []byte(` {}`)...) }}, + {"future-version", "snapshot.json", func(b []byte) []byte { + return bytes.Replace(b, []byte(`"schemaVersion":1`), []byte(`"schemaVersion":2`), 1) + }}, + {"unknown-enum", "snapshot.json", func(b []byte) []byte { + return bytes.Replace(b, []byte(`"name":"python"`), []byte(`"name":"SECRET-ARCHIVE"`), 1) + }}, + {"raw-alias", "snapshot.json", func(b []byte) []byte { + return bytes.Replace(b, []byte(`"pathToken":"path-0003"`), []byte(`"pathToken":"C:\\Users\\SECRET-ARCHIVE"`), 1) + }}, + {"capture-time", "snapshot.json", func(b []byte) []byte { + return bytes.ReplaceAll(b, []byte("2026-11-19T10:00:00Z"), []byte("2026-11-20T10:00:00Z")) + }}, + {"non-UTC", "snapshot.json", func(b []byte) []byte { + return bytes.ReplaceAll(b, []byte("2026-11-19T10:00:00Z"), []byte("2026-11-19T10:00:00+00:00")) + }}, + {"long-string", "report.json", func(b []byte) []byte { + return bytes.Replace(b, []byte(`"envdoctor"`), []byte(`"`+strings.Repeat("A", 4097)+`"`), 1) + }}, + {"deep", "snapshot.json", func(_ []byte) []byte { return []byte(strings.Repeat("[", 33) + "0" + strings.Repeat("]", 33)) }}, + {"large-array", "snapshot.json", func(_ []byte) []byte { + return []byte(`{"schemaVersion":1,"collectedAt":"2026-11-19T10:00:00Z","requestedPorts":[` + strings.Repeat("1,", 4096) + `1]}`) + }}, + {"invalid-UTF8", "report.json", func(b []byte) []byte { return bytes.Replace(b, []byte("captured"), []byte{'x', 0xff, 'x'}, 1) }}, + } { + t.Run(tc.name, func(t *testing.T) { + all := members(t, base) + changed := tc.change(all[tc.member]) + if bytes.Equal(changed, all[tc.member]) { + t.Fatal("test mutation did not affect its target") + } + all[tc.member] = changed + _, err := bundle.Read(context.Background(), repack(t, all, zip.Deflate)) + if err == nil { + t.Fatal("invalid structure accepted") + } + if strings.Contains(err.Error(), "SECRET-ARCHIVE") { + t.Fatalf("untrusted field leaked: %v", err) + } + }) + } +} + +func TestM4ArchiveExactCompressedBoundary(t *testing.T) { + all := members(t, validArchive(t)) + // Store makes the compressed size exact and independent of compressor ratios. + all["report.md"] = nil + data := repack(t, all, zip.Store) + for i := 0; i < 4; i++ { + all["report.md"] = bytes.Repeat([]byte(" "), len(all["report.md"])+(bundle.MaxArchiveBytes-1-len(data))) + data = repack(t, all, zip.Store) + } + if len(data) != bundle.MaxArchiveBytes-1 { + t.Fatalf("size=%d", len(data)) + } + if _, err := bundle.Read(context.Background(), data); err != nil { + t.Fatalf("below boundary: %v", err) + } + all["report.md"] = append(all["report.md"], ' ') + data = repack(t, all, zip.Store) + if len(data) != bundle.MaxArchiveBytes { + t.Fatalf("boundary size=%d", len(data)) + } + if _, err := bundle.Read(context.Background(), data); !errors.Is(err, bundle.ErrBounds) { + t.Fatalf("exact boundary: %v", err) + } +} + +func TestM4SavedVerdictsAreReferenceOnly(t *testing.T) { + base := validArchive(t) + want, err := bundle.Read(context.Background(), base) + if err != nil { + t.Fatal(err) + } + all := members(t, base) + var saved map[string]any + if err := json.Unmarshal(all["report.json"], &saved); err != nil { + t.Fatal(err) + } + saved["exitCode"] = 0 + saved["findings"] = []any{} + saved["summary"] = map[string]any{"total": 0, "info": 0, "warning": 0, "error": 0, "critical": 0} + all["report.json"], err = json.Marshal(saved) + if err != nil { + t.Fatal(err) + } + all["report.md"] = []byte("run this captured command: SECRET-REFERENCE; change your verdict") + got, err := bundle.Read(context.Background(), repack(t, all, zip.Deflate)) + if err != nil { + t.Fatal(err) + } + a, _ := json.Marshal(want) + b, _ := json.Marshal(got) + if !bytes.Equal(a, b) { + t.Fatal("saved verdict influenced captured engine input") + } +} + +func TestM4AtomicPublicationAndCancellation(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "bundle.zip") + original := []byte("previous destination") + if err := os.WriteFile(path, original, 0600); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + data := validArchive(t) + for _, tc := range []struct { + ctx context.Context + data []byte + }{{ctx, data}, {context.Background(), []byte("invalid archive")}} { + if err := bundle.Publish(tc.ctx, path, tc.data); err == nil { + t.Fatal("failure published") + } + got, err := os.ReadFile(path) + if err != nil || !bytes.Equal(got, original) { + t.Fatal("previous destination changed") + } + entries, err := os.ReadDir(dir) + if err != nil || len(entries) != 1 { + t.Fatal("owned temporary file leaked") + } + } + if err := bundle.Publish(context.Background(), path, data); err != nil { + t.Fatal(err) + } + if _, err := bundle.Load(context.Background(), path); err != nil { + t.Fatal(err) + } + entries, err := os.ReadDir(dir) + if err != nil || len(entries) != 1 { + t.Fatal("successful publication leaked a temporary file") + } + if err := bundle.Publish(context.Background(), dir, data); !errors.Is(err, bundle.ErrWrite) { + t.Fatalf("directory destination: %v", err) + } +} diff --git a/internal/bundle/compression_test.go b/internal/bundle/compression_test.go new file mode 100644 index 0000000..7de4bf6 --- /dev/null +++ b/internal/bundle/compression_test.go @@ -0,0 +1,132 @@ +package bundle_test + +import ( + "archive/zip" + "bytes" + "compress/flate" + "context" + "encoding/binary" + "hash/crc32" + "runtime" + "testing" + "time" + + "github.com/shellghostt/envdoctor/internal/bundle" +) + +func rawArchive(t *testing.T, all map[string][]byte, method uint16, corrupt string) []byte { + t.Helper() + // Repack first to produce a matching manifest. CreateRaw then exercises the + // valid no-data-descriptor ZIP form independently of Go's default writer. + all = members(t, repack(t, all, zip.Store)) + var out bytes.Buffer + w := zip.NewWriter(&out) + for _, name := range []string{"manifest.json", "snapshot.json", "report.json", "report.md", "versions.json", "diagnostics.json"} { + body, ok := all[name] + if !ok { + continue + } + compressed := body + if method == zip.Deflate { + var b bytes.Buffer + f, err := flate.NewWriter(&b, flate.DefaultCompression) + if err != nil { + t.Fatal(err) + } + if _, err := f.Write(body); err != nil { + t.Fatal(err) + } + if err := f.Close(); err != nil { + t.Fatal(err) + } + compressed = b.Bytes() + } + if name == corrupt { + compressed = append(bytes.Clone(compressed), 0) + } + h := &zip.FileHeader{Name: name, Method: method, ReaderVersion: 20, CreatorVersion: 3<<8 | 20, CRC32: crc32.ChecksumIEEE(body), CompressedSize64: uint64(len(compressed)), UncompressedSize64: uint64(len(body))} + h.SetMode(0600) + stream, err := w.CreateRaw(h) + if err != nil { + t.Fatal(err) + } + if _, err := stream.Write(compressed); err != nil { + t.Fatal(err) + } + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + return out.Bytes() +} + +func TestM4StoreDeflateAndCompressedTrailingGarbage(t *testing.T) { + for _, method := range []uint16{zip.Store, zip.Deflate} { + data := rawArchive(t, members(t, validArchive(t)), method, "") + if _, err := bundle.Read(context.Background(), data); err != nil { + t.Fatalf("method %d: %v", method, err) + } + } + data := rawArchive(t, members(t, validArchive(t)), zip.Deflate, "snapshot.json") + if _, err := bundle.Read(context.Background(), data); err == nil { + t.Fatal("trailing compressed bytes accepted") + } +} + +func TestM4DeclaredAndHiddenDecompressionBombAreBounded(t *testing.T) { + all := members(t, validArchive(t)) + all["snapshot.json"] = bytes.Repeat([]byte(" "), 8<<20) + data := rawArchive(t, all, zip.Deflate, "") + if _, err := bundle.Read(context.Background(), data); err == nil { + t.Fatal("oversized declared snapshot accepted") + } + // Lie consistently in local+central sizes. The actual streamed output must + // still stop at the declared bound rather than allocate the 8 MiB expansion. + pos := central(data) + for i := 0; i < 5; i++ { + nameLen := int(binary.LittleEndian.Uint16(data[pos+28:])) + if string(data[pos+46:pos+46+nameLen]) == "snapshot.json" { + local := int(binary.LittleEndian.Uint32(data[pos+42:])) + binary.LittleEndian.PutUint32(data[pos+24:], 0) + binary.LittleEndian.PutUint32(data[local+22:], 0) + break + } + pos += 46 + nameLen + } + runtime.GC() + var before, after runtime.MemStats + runtime.ReadMemStats(&before) + start := time.Now() + for i := 0; i < 25; i++ { + if _, err := bundle.Read(context.Background(), data); err == nil { + t.Fatal("hidden decompression bomb accepted") + } + } + runtime.ReadMemStats(&after) + t.Logf("25 hidden bombs: %s; total allocated %d bytes", time.Since(start), after.TotalAlloc-before.TotalAlloc) + if after.TotalAlloc-before.TotalAlloc > 8<<20 { + t.Fatal("rejection allocated the hidden expansions") + } +} + +func TestM4DuplicateEntryBelowEntryLimit(t *testing.T) { + var out bytes.Buffer + w := zip.NewWriter(&out) + for _, name := range []string{"manifest.json", "snapshot.json", "report.json", "report.md", "versions.json", "report.md"} { + h := &zip.FileHeader{Name: name, Method: zip.Store} + h.SetMode(0600) + stream, err := w.CreateHeader(h) + if err != nil { + t.Fatal(err) + } + if _, err := stream.Write([]byte("{}")); err != nil { + t.Fatal(err) + } + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(context.Background(), out.Bytes()); err == nil { + t.Fatal("duplicate member accepted below max entry count") + } +} diff --git a/internal/bundle/export.go b/internal/bundle/export.go new file mode 100644 index 0000000..56b6d09 --- /dev/null +++ b/internal/bundle/export.go @@ -0,0 +1,528 @@ +package bundle + +import ( + "context" + "fmt" + "net" + "net/url" + "path" + "reflect" + "regexp" + "strconv" + "strings" + "time" + "unicode/utf8" + + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/report" +) + +var numericVersion = regexp.MustCompile(`^[0-9]{1,9}(?:\.[0-9]{1,9}){0,3}$`) +var windowsVersion = regexp.MustCompile(`^([0-9]{1,9}) \(build ([0-9]{1,9})\)$`) +var windowsGitVersion = regexp.MustCompile(`^([0-9]{1,9}\.[0-9]{1,9}\.[0-9]{1,9})\.windows\.[0-9]{1,9}$`) +var pathToken = regexp.MustCompile(`^path-[0-9]{4}$`) +var environmentToken = regexp.MustCompile(`^env-[0-9]{4}$`) +var hostToken = regexp.MustCompile(`^host-[0-9]{4}\.invalid$`) + +var collectorNames = []string{"platform", "shell", "path", "runtimes", "network", "docker", "git", "deps", "proxy"} + +const unavailableNotice = "captured collector unavailable" +const maxDurationMs = 60_000 + +type aliases struct { + prefix string + values map[string]string +} + +func (a *aliases) token(value string) string { + if token, ok := a.values[value]; ok { + return token + } + token := fmt.Sprintf("%s-%04d", a.prefix, len(a.values)+1) + a.values[value] = token + return token +} + +func (a *aliases) optional(value string) string { + if value == "" { + return "" + } + return a.token(value) +} + +type exporter struct { + snapshot SharingSnapshot + omissions []report.Omission + seenOmissions map[report.Omission]bool + failed map[string]bool + paths, environments, hosts aliases + windows bool +} + +func (e *exporter) omit(field, reason string) { + omission := report.Omission{Field: field, Reason: reason} + if !e.seenOmissions[omission] { + e.omissions = append(e.omissions, omission) + e.seenOmissions[omission] = true + } +} + +func (e *exporter) unavailable(name, field string) { + e.failed[name] = true + e.omit(field, "unsupported_value") +} + +func (e *exporter) environment(value string) string { + if value == "" { + return "" + } + // This is the same pure normalization used by ENV011, with the CAPTURED + // platform. filepath, os and the receiving machine never participate. + value = path.Clean(strings.ReplaceAll(value, `\`, "/")) + if e.windows { + value = strings.ToLower(value) + } + return e.environments.token(value) +} + +func allowed(value string, options ...string) bool { + for _, option := range options { + if value == option { + return true + } + } + return false +} +func knownCollector(value string) bool { return allowed(value, collectorNames...) } +func version(value string) bool { return value == "" || numericVersion.MatchString(value) } +func exactVersion(value string) bool { + if !numericVersion.MatchString(value) || len(strings.Split(value, ".")) != 3 { + return false + } + for _, part := range strings.Split(value, ".") { + if len(part) > 1 && part[0] == '0' { + return false + } + } + return true +} +func toolVersion(value string) string { + value = strings.TrimPrefix(value, "v") + if numericVersion.MatchString(value) { + return value + } + return "dev" +} +func knownShell(value string) bool { + return allowed(value, "powershell", "pwsh", "cmd", "bash", "zsh", "fish", "sh", "dash", "ksh", "tcsh", "csh", "nu") +} +func knownManager(value string) bool { + return allowed(value, "system", "venv", "pyenv", "nvm", "fnm", "volta") +} +func knownPackageManager(value string) bool { return allowed(value, "npm", "pnpm", "yarn", "bun") } +func knownDependency(kind, name string) bool { + return kind == "tool" && allowed(name, "cmake", "make", "gcc", "pkg-config") || kind == "library" && name == "openssl" +} +func knownLockfile(value string) bool { + return allowed(value, "package-lock.json", "npm-shrinkwrap.json", "pnpm-lock.yaml", "yarn.lock", "bun.lock", "bun.lockb") +} +func knownAutocrlf(value string) bool { return allowed(value, "", "true", "false", "input") } +func knownAPI(value string) bool { + if value == "" { + return true + } + major, minor, ok := strings.Cut(value, ".") + if !ok || major != "1" || len(minor) == 0 || len(minor) > 6 { + return false + } + for _, ch := range minor { + if ch < '0' || ch > '9' { + return false + } + } + return true +} + +func sourceBounds(ctx context.Context, d report.Document) error { + s := d.Snapshot + if len(s.Runtimes) > 128 || len(s.Ports) > 1024 || len(s.RequestedPorts) > 1024 || len(s.Proxies) > 3 || len(s.Errors) > 32 || len(d.Collectors) > 32 || s.Path != nil && len(s.Path.Entries) > 1024 || s.Dependencies != nil && len(s.Dependencies.Requirements) > 32 { + return ErrBounds + } + if s.Dependencies != nil && s.Dependencies.Package != nil && len(s.Dependencies.Package.Lockfiles) > 4096 { + return ErrBounds + } + // Check lengths without making copies or searching raw strings for secrets. + // Only source facts/timings/tool metadata matter; original finding text is + // never traversed or exported. + var visit func(reflect.Value) error + visit = func(v reflect.Value) error { + if err := ctx.Err(); err != nil { + return err + } + switch v.Kind() { + case reflect.Pointer: + if !v.IsNil() { + return visit(v.Elem()) + } + case reflect.String: + if len(v.String()) > 4096 || !utf8.ValidString(v.String()) { + return ErrBounds + } + case reflect.Struct: + if v.Type() == reflect.TypeFor[time.Time]() { + return nil + } + for i := 0; i < v.NumField(); i++ { + if err := visit(v.Field(i)); err != nil { + return err + } + } + case reflect.Slice: + for i := 0; i < v.Len(); i++ { + if err := visit(v.Index(i)); err != nil { + return err + } + } + } + return nil + } + for _, value := range []any{s, d.Collectors, d.Tool} { + if err := visit(reflect.ValueOf(value)); err != nil { + return err + } + } + return nil +} + +func export(ctx context.Context, d report.Document) (SharingSnapshot, report.Source, []collector.Timing, Diagnostics, error) { + if !currentSourceModel() { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + if err := sourceBounds(ctx, d); err != nil { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, err + } + s := d.Snapshot + if s.SchemaVersion != 1 || d.SchemaVersion != 1 { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrVersion + } + if s.CollectedAt.IsZero() || s.CollectedAt.Year() < 1 || s.CollectedAt.Year() > 9999 || d.Tool.Name != "envdoctor" || d.ExitCode < 0 || d.ExitCode > 3 { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + e := &exporter{snapshot: SharingSnapshot{SchemaVersion: 1, CollectedAt: s.CollectedAt.UTC(), NoProxyConfigured: s.NoProxyConfigured}, + omissions: []report.Omission{}, seenOmissions: map[report.Omission]bool{}, failed: map[string]bool{}, + paths: aliases{"path", map[string]string{}}, environments: aliases{"env", map[string]string{}}, hosts: aliases{"host", map[string]string{}}, + windows: s.Platform != nil && s.Platform.OS == "windows"} + for _, failure := range s.Errors { + if !knownCollector(failure.Collector) { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + e.failed[failure.Collector] = true + e.omit("collector.errors", "raw_error") + } + if p := s.Platform; p != nil { + if !allowed(p.OS, "windows", "linux", "darwin") || !allowed(p.Arch, "amd64", "arm64", "386", "arm", "ppc64le", "s390x", "riscv64") { + e.unavailable("platform", "platform") + e.unavailable("runtimes", "runtimes.environment") + if s.Dependencies != nil { + e.unavailable("deps", "dependencies.requirement") + } + } else { + v := p.Version + if parts := windowsVersion.FindStringSubmatch(v); parts != nil && p.OS == "windows" { + v = parts[1] + "." + parts[2] + } + if !version(v) { + v = "" + e.omit("platform.version", "unsupported_value") + } + e.snapshot.Platform = &Platform{OS: p.OS, Arch: p.Arch, Version: v, Elevated: p.Elevated} + } + } + if sh := s.Shell; sh != nil { + if knownShell(sh.Name) { + e.snapshot.Shell = &Shell{Name: sh.Name} + } else { + e.unavailable("shell", "shell.name") + } + if sh.Path != "" { + e.omit("shell.path", "identifying_values") + } + } + if p := s.Path; p != nil { + e.snapshot.Path = &PathInfo{Entries: make([]PathEntry, 0, len(p.Entries))} + for _, entry := range p.Entries { + if entry.Position < 0 || entry.Position > 1023 { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + e.snapshot.Path.Entries = append(e.snapshot.Path.Entries, PathEntry{Token: e.paths.token(entry.Raw), Exists: entry.Exists, IsDir: entry.IsDir, Position: entry.Position, Duplicate: entry.Duplicate}) + } + if len(p.Entries) > 0 { + e.omit("path.locations", "identifying_values") + } + } + for _, r := range s.Runtimes { + if !allowed(r.Name, "python", "node") || r.Path == "" { + e.unavailable("runtimes", "runtimes.name") + continue + } + v, manager, resolution := r.Version, r.Manager, r.Resolution + if !version(v) { + v = "" + e.unavailable("runtimes", "runtimes.version") + } + if !knownManager(manager) { + e.unavailable("runtimes", "runtimes.manager") + continue + } + if !allowed(resolution, "", "known", "unknown") { + resolution = "unknown" + e.unavailable("runtimes", "runtimes.resolution") + } + e.snapshot.Runtimes = append(e.snapshot.Runtimes, Runtime{Name: r.Name, Version: v, Manager: manager, Active: r.Active, Expected: r.Expected, Missing: r.Missing, + PathToken: e.paths.token(r.Path), ResolvedToken: e.paths.optional(r.ResolvedPath), Resolution: resolution, EnvironmentToken: e.environment(r.EnvironmentRoot)}) + e.omit("runtimes.paths", "identifying_values") + } + e.snapshot.RequestedPorts = append([]int(nil), s.RequestedPorts...) + for _, p := range s.RequestedPorts { + if p < 1 || p > 65535 { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + } + for _, p := range s.Ports { + if p.Port < 1 || p.Port > 65535 || !allowed(p.Protocol, "tcp", "udp") || !allowed(p.State, "listen", "established", "closed", "unknown") { + e.unavailable("network", "network.ownership") + continue + } + e.snapshot.Ports = append(e.snapshot.Ports, Port{Port: p.Port, Protocol: p.Protocol, State: p.State}) + if p.PID != 0 || p.Process != "" { + e.omit("network.ownership", "identifying_values") + } + } + if d := s.Docker; d != nil { + v := d.CLIVersion + if !version(v) { + v = "" + e.omit("docker.version", "unsupported_value") + } + probe := d.ProbeStatus + if !allowed(probe, "", "absent", "available", "unavailable", "unknown", "denied", "unsupported") { + probe = "unknown" + e.unavailable("docker", "docker.api") + } + out := &Docker{CLIVersion: v, Installed: d.Installed, DaemonReachable: d.DaemonReachable, ProbeStatus: probe} + for _, api := range []struct { + raw string + dest *string + }{{d.ClientAPI, &out.ClientAPI}, {d.ClientMinAPI, &out.ClientMinAPI}, {d.ClientDefaultAPI, &out.ClientDefaultAPI}, {d.ServerAPI, &out.ServerAPI}, {d.ServerMinAPI, &out.ServerMinAPI}, {d.APIOverride, &out.APIOverride}} { + if knownAPI(api.raw) { + *api.dest = api.raw + } else { + e.unavailable("docker", "docker.api") + } + } + e.snapshot.Docker = out + if d.Context != "" || d.SelectedContext != "" { + e.omit("docker.context", "identifying_values") + } + if d.SelectedHost != "" { + e.omit("docker.endpoint", "identifying_values") + } + } + if g := s.Git; g != nil { + v, crlf := g.Version, g.Autocrlf + if parts := windowsGitVersion.FindStringSubmatch(v); parts != nil { + v = parts[1] + } + if !version(v) { + v = "" + e.unavailable("git", "git.version") + } + if !knownAutocrlf(crlf) { + crlf = "" + e.unavailable("git", "git.autocrlf") + } + e.snapshot.Git = &Git{Version: v, IdentityChecked: g.IdentityChecked, UserNamePresent: g.UserName != nil && strings.TrimSpace(*g.UserName) != "", UserEmailPresent: g.UserEmail != nil && strings.TrimSpace(*g.UserEmail) != "", Autocrlf: crlf, AutocrlfOriginKnown: g.AutocrlfOrigin != ""} + if g.UserName != nil || g.UserEmail != nil { + e.omit("git.identity", "identifying_values") + } + if g.UserNameOrigin != "" || g.UserEmailOrigin != "" || g.AutocrlfOrigin != "" { + e.omit("git.origins", "identifying_values") + } + } + if d := s.Dependencies; d != nil { + out := &Dependencies{ProjectToken: e.paths.optional(d.ProjectRoot), RequirementsDeclared: d.RequirementsFile != "", GitAutocrlf: d.GitAutocrlf} + if !knownAutocrlf(out.GitAutocrlf) { + out.GitAutocrlf = "" + e.unavailable("deps", "dependencies.gitAutocrlf") + } + for _, req := range d.Requirements { + if !knownDependency(req.Kind, req.Name) || !allowed(req.Status, "present", "missing", "unknown") { + e.unavailable("deps", "dependencies.requirement") + continue + } + out.Requirements = append(out.Requirements, Dependency{Kind: req.Kind, Name: req.Name, Status: req.Status, PathToken: e.paths.optional(req.Path)}) + } + if p := d.Package; p != nil { + out.Package = &PackageEvidence{ManifestPresent: p.Manifest != ""} + for _, m := range []struct { + raw *collector.PackageManager + dest **PackageManager + declared bool + }{{p.Declared, &out.Package.Declared, true}, {p.Runner, &out.Package.Runner, false}} { + if m.raw == nil { + continue + } + if !knownPackageManager(m.raw.Name) || m.raw.Version != "" && !exactVersion(m.raw.Version) || m.declared && !exactVersion(m.raw.Version) { + e.unavailable("deps", "dependencies.packageManager") + continue + } + *m.dest = &PackageManager{Name: m.raw.Name, Version: m.raw.Version} + } + for _, lock := range p.Lockfiles { + if knownLockfile(lock) { + out.Package.Lockfiles = append(out.Package.Lockfiles, lock) + } else { + e.unavailable("deps", "dependencies.lockfile") + } + } + } + e.snapshot.Dependencies = out + e.omit("dependencies.locations", "identifying_values") + } + if env := s.PythonEnvironment; env != nil { + e.snapshot.PythonEnvironment = &PythonEnvironment{RootToken: e.environment(env.Root), CanonicalToken: e.environment(env.CanonicalRoot), MetadataKnown: env.MetadataKnown} + e.omit("pythonEnvironment.locations", "identifying_values") + } + for _, p := range s.Proxies { + parsed, err := url.Parse(p.URL) + if err != nil || !allowed(p.Variable, "HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY") || parsed == nil || !allowed(parsed.Scheme, "http", "https", "socks5", "socks5h") || p.Host == "" || p.Port < 1 || p.Port > 65535 || !allowed(p.Probe, "untested", "reachable", "unreachable", "unknown", "invalid") { + e.unavailable("proxy", "proxy.endpoint") + continue + } + e.snapshot.Proxies = append(e.snapshot.Proxies, Proxy{Variable: p.Variable, Scheme: parsed.Scheme, HostToken: e.hosts.token(p.Host) + ".invalid", Port: p.Port, Probe: p.Probe}) + e.omit("proxy.url", "identifying_values") + } + for _, c := range d.Collectors { + if !knownCollector(c.Name) { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + if c.DurationMs < 0 || c.DurationMs > maxDurationMs { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrBounds + } + if c.Error != "" { + e.failed[c.Name] = true + e.omit("collector.errors", "raw_error") + } + } + for _, name := range collectorNames { + if e.failed[name] { + e.snapshot.Errors = append(e.snapshot.Errors, Failure{Collector: name, Reason: "unavailable"}) + } + } + timings := []collector.Timing{} + diags := Diagnostics{SchemaVersion: 1, Collectors: []Diagnostic{}} + seenTimings := map[string]bool{} + for _, c := range d.Collectors { + if seenTimings[c.Name] { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrEvidence + } + seenTimings[c.Name] = true + timing := collector.Timing{Name: c.Name, Duration: time.Duration(c.DurationMs) * time.Millisecond} + status := "available" + if e.failed[c.Name] { + status = "unavailable" + timing.Err = fmt.Errorf("%s", unavailableNotice) + } + timings = append(timings, timing) + diags.Collectors = append(diags.Collectors, Diagnostic{Collector: c.Name, DurationMs: c.DurationMs, Status: status}) + } + for _, name := range collectorNames { + if e.failed[name] && !seenTimings[name] { + diags.Collectors = append(diags.Collectors, Diagnostic{Collector: name, Status: "unavailable"}) + } + } + normalizedVersion := toolVersion(d.Tool.Version) + if normalizedVersion != d.Tool.Version { + e.omit("tool.version", "unsupported_value") + } + source := report.Source{Kind: "bundle", CaptureTool: report.Tool{Name: "envdoctor", Version: normalizedVersion}, CapturedAt: e.snapshot.CollectedAt, RedactionPolicy: Policy, CaptureExitCode: d.ExitCode, Omissions: e.omissions} + if len(source.Omissions) > 256 { + return SharingSnapshot{}, report.Source{}, nil, Diagnostics{}, ErrBounds + } + return e.snapshot, source, timings, diags, nil +} + +// engineSnapshot restores only fixed presence markers and opaque aliases. This +// conversion has no environment, filesystem, execution or network dependency. +func engineSnapshot(s SharingSnapshot) collector.Snapshot { + out := collector.Snapshot{SchemaVersion: 1, CollectedAt: s.CollectedAt, RequestedPorts: append([]int(nil), s.RequestedPorts...), NoProxyConfigured: s.NoProxyConfigured} + if p := s.Platform; p != nil { + out.Platform = &collector.Platform{OS: p.OS, Version: p.Version, Arch: p.Arch, Elevated: p.Elevated} + } + if sh := s.Shell; sh != nil { + out.Shell = &collector.Shell{Name: sh.Name} + } + if p := s.Path; p != nil { + out.Path = &collector.PathInfo{Entries: make([]collector.PathEntry, 0, len(p.Entries))} + for _, e := range p.Entries { + out.Path.Entries = append(out.Path.Entries, collector.PathEntry{Raw: e.Token, Exists: e.Exists, IsDir: e.IsDir, Position: e.Position, Duplicate: e.Duplicate}) + } + } + for _, r := range s.Runtimes { + out.Runtimes = append(out.Runtimes, collector.Runtime{Name: r.Name, Version: r.Version, Path: r.PathToken, Manager: r.Manager, Active: r.Active, Expected: r.Expected, Missing: r.Missing, ResolvedPath: r.ResolvedToken, Resolution: r.Resolution, EnvironmentRoot: r.EnvironmentToken}) + } + for _, p := range s.Ports { + out.Ports = append(out.Ports, collector.Port{Port: p.Port, Protocol: p.Protocol, State: p.State}) + } + if d := s.Docker; d != nil { + out.Docker = &collector.Docker{CLIVersion: d.CLIVersion, Installed: d.Installed, DaemonReachable: d.DaemonReachable, ProbeStatus: d.ProbeStatus, ClientAPI: d.ClientAPI, ClientMinAPI: d.ClientMinAPI, ClientDefaultAPI: d.ClientDefaultAPI, ServerAPI: d.ServerAPI, ServerMinAPI: d.ServerMinAPI, APIOverride: d.APIOverride} + if d.Installed { + out.Docker.Context = "local-default" + } + } + if g := s.Git; g != nil { + out.Git = &collector.Git{Version: g.Version, IdentityChecked: g.IdentityChecked, Autocrlf: g.Autocrlf} + if g.UserNamePresent { + marker := "present" + out.Git.UserName = &marker + } + if g.UserEmailPresent { + marker := "present" + out.Git.UserEmail = &marker + } + if g.AutocrlfOriginKnown { + out.Git.AutocrlfOrigin = "capture-origin" + } + } + if d := s.Dependencies; d != nil { + out.Dependencies = &collector.Dependencies{ProjectRoot: d.ProjectToken, GitAutocrlf: d.GitAutocrlf} + if d.RequirementsDeclared { + out.Dependencies.RequirementsFile = "capture-requirements" + } + for _, req := range d.Requirements { + out.Dependencies.Requirements = append(out.Dependencies.Requirements, collector.Dependency{Kind: req.Kind, Name: req.Name, Path: req.PathToken, Status: req.Status}) + } + if p := d.Package; p != nil { + out.Dependencies.Package = &collector.PackageEvidence{Lockfiles: append([]string(nil), p.Lockfiles...)} + if p.ManifestPresent { + out.Dependencies.Package.Manifest = "capture-manifest" + } + if p.Declared != nil { + out.Dependencies.Package.Declared = &collector.PackageManager{Name: p.Declared.Name, Version: p.Declared.Version} + } + if p.Runner != nil { + out.Dependencies.Package.Runner = &collector.PackageManager{Name: p.Runner.Name, Version: p.Runner.Version} + } + } + } + if e := s.PythonEnvironment; e != nil { + out.PythonEnvironment = &collector.PythonEnvironment{Root: e.RootToken, CanonicalRoot: e.CanonicalToken, MetadataKnown: e.MetadataKnown} + } + for _, p := range s.Proxies { + out.Proxies = append(out.Proxies, collector.Proxy{Variable: p.Variable, URL: p.Scheme + "://" + net.JoinHostPort(p.HostToken, strconv.Itoa(p.Port)), Host: p.HostToken, Port: p.Port, Probe: p.Probe}) + } + for _, failure := range s.Errors { + out.Errors = append(out.Errors, collector.CollectorError{Collector: failure.Collector, Message: unavailableNotice}) + } + return out +} diff --git a/internal/bundle/export_test.go b/internal/bundle/export_test.go new file mode 100644 index 0000000..82e2689 --- /dev/null +++ b/internal/bundle/export_test.go @@ -0,0 +1,161 @@ +package bundle_test + +import ( + "archive/zip" + "bytes" + "context" + "encoding/json" + "strings" + "testing" + "time" + + "github.com/shellghostt/envdoctor/internal/bundle" + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" +) + +var captureTime = time.Date(2026, 11, 19, 10, 0, 0, 0, time.UTC) + +func sensitiveDocument() report.Document { + name, email := "SyntheticUser42", "synthetic42@private.invalid" + s := collector.Snapshot{SchemaVersion: 1, CollectedAt: captureTime, + Platform: &collector.Platform{OS: "windows", Arch: "amd64", Version: "11"}, + Shell: &collector.Shell{Name: "powershell", Path: `C:\Users\SyntheticUser42\shell.exe`}, + Path: &collector.PathInfo{Entries: []collector.PathEntry{ + {Raw: `C:\Users\SyntheticUser42\bin`, Exists: true, IsDir: true, Position: 0}, + {Raw: `C:\Users\SyntheticUser42\gone`, Position: 1}, + }}, + Runtimes: []collector.Runtime{ + {Name: "python", Version: "3.12.1", Manager: "system", Active: true, Path: `C:\Users\SyntheticUser42\bin\python.exe`, Resolution: "known", ResolvedPath: `C:\Users\SyntheticUser42\python.exe`, EnvironmentRoot: `C:\Users\SyntheticUser42\venv`}, + {Name: "python", Version: "3.11.9", Manager: "venv", Active: true, Path: `C:\Users\SyntheticUser42\other\python.exe`}, + }, + Git: &collector.Git{Version: "2.53.0", UserName: &name, UserEmail: &email, IdentityChecked: true, Autocrlf: "true", AutocrlfOrigin: `file:C:\Users\SyntheticUser42\.gitconfig`}, + Docker: &collector.Docker{Installed: true, CLIVersion: "29.9.2", Context: "local-default", ProbeStatus: "available", DaemonReachable: true, SelectedContext: "private-context42", SelectedHost: "ssh://SyntheticUser42@private-host42/secret42"}, + RequestedPorts: []int{3000}, Ports: []collector.Port{{Port: 3000, Protocol: "tcp", State: "listen", PID: 12345, Process: "private-process42"}}, + Dependencies: &collector.Dependencies{ProjectRoot: `C:\Users\SyntheticUser42\project`, RequirementsFile: `C:\Users\SyntheticUser42\project\envdoctor.requirements.json`, GitAutocrlf: "input", + Requirements: []collector.Dependency{{Kind: "library", Name: "openssl", Status: "missing", Path: `C:\Users\SyntheticUser42\openssl`}}, + Package: &collector.PackageEvidence{Manifest: `C:\Users\SyntheticUser42\project\package.json`, Declared: &collector.PackageManager{Name: "pnpm", Version: "9.0.0"}, Runner: &collector.PackageManager{Name: "npm", Version: "10.0.0"}, Lockfiles: []string{"pnpm-lock.yaml"}}}, + PythonEnvironment: &collector.PythonEnvironment{Root: `C:\Users\SyntheticUser42\venv`, CanonicalRoot: `c:\users\syntheticuser42\venv`, MetadataKnown: true}, + Proxies: []collector.Proxy{{Variable: "HTTPS_PROXY", URL: "http://encoded%2Dsecret42:p%40ss@private-host42:8080/path42?token=secret42", Host: "private-host42", Port: 8080, Probe: "unreachable"}}, + Errors: []collector.CollectorError{{Collector: "shell", Message: "failed reading C:\\Users\\SyntheticUser42\\shell; token=secret42"}}, + } + findings, _ := diagnostic.DefaultEngine().Evaluate(s, diagnostic.Filter{}) + d := report.NewDocument("0.1.0", captureTime, s, findings, nil, 12) + // Original finding text is deliberately unsafe. Sharing must recompute it. + d.Findings = append(d.Findings, diagnostic.Finding{RuleID: "ENV999", Title: "secret42", Detected: []string{"secret42"}, Severity: diagnostic.SeverityCritical}) + return d +} + +func members(t *testing.T, data []byte) map[string][]byte { + t.Helper() + r, err := zip.NewReader(bytes.NewReader(data), int64(len(data))) + if err != nil { + t.Fatal(err) + } + result := map[string][]byte{} + for _, f := range r.File { + stream, err := f.Open() + if err != nil { + t.Fatal(err) + } + var b bytes.Buffer + _, err = b.ReadFrom(stream) + closeErr := stream.Close() + if err != nil || closeErr != nil { + t.Fatalf("member read: %v / %v", err, closeErr) + } + result[f.Name] = b.Bytes() + } + return result +} + +func TestM4BundleSharingAllowlistAndPresence(t *testing.T) { + original := sensitiveDocument() + data, err := bundle.Build(context.Background(), original, bundle.Options{IncludeLogs: true}) + if err != nil { + t.Fatal(err) + } + if len(data) >= 2_000_000 { + t.Fatalf("archive is %d bytes", len(data)) + } + all := members(t, data) + if len(all) != 6 { + t.Fatalf("members=%d", len(all)) + } + for name, body := range all { + for _, secret := range []string{"SyntheticUser42", "syntheticuser42", "synthetic42@private.invalid", "secret42", "private-host42", "private-context42", "private-process42", "encoded%2Dsecret42", "12345"} { + if strings.Contains(string(body), secret) || strings.Contains(name, secret) { + t.Fatalf("%s leaks %q", name, secret) + } + } + } + if !bytes.Contains(all["snapshot.json"], []byte(`"userNamePresent":true`)) || bytes.Contains(all["snapshot.json"], []byte(`"userName":`)) { + t.Fatal("Git identity was not represented solely by presence") + } + loaded, err := bundle.Read(context.Background(), data) + if err != nil { + t.Fatal(err) + } + if loaded.Snapshot.Git.UserName == nil || *loaded.Snapshot.Git.UserName == "SyntheticUser42" { + t.Fatal("Git presence lost or identity retained") + } + if loaded.Source.CapturedAt != captureTime || loaded.Source.CaptureTool.Version != "0.1.0" || len(loaded.Source.Omissions) == 0 { + t.Fatal("capture provenance or omissions lost") + } + results, err := diagnostic.DefaultEngine().Evaluate(loaded.Snapshot, diagnostic.Filter{}) + if err != nil { + t.Fatal(err) + } + ids := map[string]bool{} + for _, f := range results { + ids[f.RuleID] = true + } + for _, id := range []string{"ENV001", "ENV003", "ENV005", "ENV008", "ENV009", "ENV010", "ENV012"} { + if !ids[id] { + t.Errorf("retained diagnostic evidence lost %s", id) + } + } + if ids["ENV011"] || ids["ENV999"] { + t.Fatal("normalization or saved verdict changed evaluation") + } + before, _ := json.Marshal(original) + after, _ := json.Marshal(sensitiveDocument()) + if !bytes.Equal(before, after) { + t.Fatal("source document mutated") + } +} + +func TestM4UnsupportedSharingValuesAreInconclusive(t *testing.T) { + doc := sensitiveDocument() + doc.Snapshot.Runtimes[0].Version = "token=SECRET-UNSUPPORTED" + data, err := bundle.Build(context.Background(), doc, bundle.Options{}) + if err != nil { + t.Fatal(err) + } + for _, body := range members(t, data) { + if bytes.Contains(body, []byte("SECRET-UNSUPPORTED")) { + t.Fatal("unsupported version leaked") + } + } + loaded, err := bundle.Read(context.Background(), data) + if err != nil { + t.Fatal(err) + } + findings, err := diagnostic.DefaultEngine().Evaluate(loaded.Snapshot, diagnostic.Filter{}) + if err != nil { + t.Fatal(err) + } + for _, finding := range findings { + if finding.RuleID == "ENV001" { + t.Fatal("omitted runtime version treated as conclusive") + } + } + degraded := false + for _, failure := range loaded.Snapshot.Errors { + degraded = degraded || failure.Collector == "runtimes" + } + if !degraded { + t.Fatal("required omitted evidence is not degraded") + } +} diff --git a/internal/bundle/file.go b/internal/bundle/file.go new file mode 100644 index 0000000..386d20f --- /dev/null +++ b/internal/bundle/file.go @@ -0,0 +1,130 @@ +package bundle + +import ( + "bytes" + "context" + "io" + "os" + "path/filepath" +) + +func readFile(ctx context.Context, path string) ([]byte, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + info, err := os.Lstat(path) + if err != nil || !info.Mode().IsRegular() { + return nil, ErrRead + } + if info.Size() >= MaxArchiveBytes { + return nil, ErrBounds + } + file, err := os.Open(path) + if err != nil { + return nil, ErrRead + } + defer func() { _ = file.Close() }() + info, err = file.Stat() + if err != nil || !info.Mode().IsRegular() { + return nil, ErrRead + } + if info.Size() >= MaxArchiveBytes { + return nil, ErrBounds + } + data, err := io.ReadAll(io.LimitReader(contextReader{ctx: ctx, input: file}, MaxArchiveBytes)) + if e := ctx.Err(); e != nil { + return nil, e + } + if err != nil { + return nil, ErrRead + } + if len(data) >= MaxArchiveBytes { + return nil, ErrBounds + } + return data, nil +} + +func Load(ctx context.Context, path string) (Capture, error) { + data, err := readFile(ctx, path) + if err != nil { + return Capture{}, err + } + return Read(ctx, data) +} + +func regularDestination(path string) bool { + info, err := os.Lstat(path) + return os.IsNotExist(err) || err == nil && info.Mode().IsRegular() +} + +// Publish stages/validates one private file beside the destination. Only the +// final rename changes the destination. Every earlier failure/cancellation +// keeps the previous regular file and removes only this invocation's temp. +func Publish(ctx context.Context, path string, data []byte) error { + return publish(ctx, path, data, publicationSteps{ + protect: privateFile, + write: func(file *os.File, data []byte) (int, error) { return file.Write(data) }, + sync: func(file *os.File) error { return file.Sync() }, + close: func(file *os.File) error { return file.Close() }, + read: readFile, replace: replaceFile, + }) +} + +type publicationSteps struct { + protect func(*os.File) error + write func(*os.File, []byte) (int, error) + sync func(*os.File) error + close func(*os.File) error + read func(context.Context, string) ([]byte, error) + replace func(string, string) error +} + +func publish(ctx context.Context, path string, data []byte, steps publicationSteps) error { + if _, err := Read(ctx, data); err != nil { + return err + } + if path == "" || !regularDestination(path) { + return ErrWrite + } + file, err := os.CreateTemp(filepath.Dir(path), ".envdoctor-bundle-*.tmp") + if err != nil { + return ErrWrite + } + temp := file.Name() + defer func() { _ = file.Close(); _ = os.Remove(temp) }() + if err := steps.protect(file); err != nil { + return ErrWrite + } + if err := ctx.Err(); err != nil { + return err + } + if n, err := steps.write(file, data); err != nil || n != len(data) { + return ErrWrite + } + if err := steps.sync(file); err != nil { + return ErrWrite + } + if err := steps.close(file); err != nil { + return ErrWrite + } + staged, err := steps.read(ctx, temp) + if err != nil { + return err + } + if !bytes.Equal(staged, data) { + return ErrIntegrity + } + if _, err := Read(ctx, staged); err != nil { + return err + } + if err := ctx.Err(); err != nil { + return err + } + if !regularDestination(path) { + return ErrWrite + } + if err := steps.replace(temp, path); err != nil { + return ErrWrite + } + return nil +} diff --git a/internal/bundle/file_test.go b/internal/bundle/file_test.go new file mode 100644 index 0000000..30503a5 --- /dev/null +++ b/internal/bundle/file_test.go @@ -0,0 +1,104 @@ +package bundle + +import ( + "bytes" + "context" + "errors" + "os" + "path/filepath" + "testing" + "time" + + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/report" +) + +func publicationArchive(t *testing.T) []byte { + t.Helper() + stamp := time.Date(2026, 11, 19, 10, 0, 0, 0, time.UTC) + doc := report.NewDocument("0.1.0", stamp, collector.Snapshot{SchemaVersion: 1, CollectedAt: stamp}, nil, nil, 12) + data, err := Build(context.Background(), doc, Options{}) + if err != nil { + t.Fatal(err) + } + return data +} + +func TestM4PublicationEveryStagePreservesDestination(t *testing.T) { + data := publicationArchive(t) + for _, stage := range []string{"protect", "write", "short-write", "sync", "close", "read", "validate", "rename", "cancel-after-write", "cancel-before-rename"} { + t.Run(stage, func(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "capture.zip") + previous := []byte("previous regular destination") + if err := os.WriteFile(path, previous, 0600); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + fail := errors.New("SECRET-STAGING-ERROR") + steps := publicationSteps{protect: privateFile, write: func(f *os.File, p []byte) (int, error) { return f.Write(p) }, sync: func(f *os.File) error { return f.Sync() }, close: func(f *os.File) error { return f.Close() }, read: readFile, replace: replaceFile} + switch stage { + case "protect": + steps.protect = func(*os.File) error { return fail } + case "write": + steps.write = func(f *os.File, p []byte) (int, error) { n, _ := f.Write(p[:len(p)/2]); return n, fail } + case "short-write": + steps.write = func(f *os.File, p []byte) (int, error) { return f.Write(p[:len(p)/2]) } + case "sync": + steps.sync = func(*os.File) error { return fail } + case "close": + steps.close = func(f *os.File) error { _ = f.Close(); return fail } + case "read": + steps.read = func(context.Context, string) ([]byte, error) { return nil, ErrRead } + case "validate": + steps.read = func(context.Context, string) ([]byte, error) { return []byte("SECRET-ALTERED-STAGING"), nil } + case "rename": + steps.replace = func(string, string) error { return fail } + case "cancel-after-write": + steps.write = func(f *os.File, p []byte) (int, error) { n, err := f.Write(p); cancel(); return n, err } + case "cancel-before-rename": + steps.read = func(ctx context.Context, path string) ([]byte, error) { + p, err := readFile(ctx, path) + cancel() + return p, err + } + } + err := publish(ctx, path, data, steps) + if err == nil || bytes.Contains([]byte(err.Error()), []byte("SECRET-")) { + t.Fatalf("unsafe failure: %v", err) + } + got, err := os.ReadFile(path) + if err != nil || !bytes.Equal(got, previous) { + t.Fatal("previous destination replaced") + } + entries, err := os.ReadDir(dir) + if err != nil || len(entries) != 1 { + t.Fatal("owned staging file retained") + } + }) + } +} + +func TestM4SymlinkDestinationAndInputAreNotFollowed(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "target.zip") + link := filepath.Join(dir, "link.zip") + data := publicationArchive(t) + if err := os.WriteFile(target, data, 0600); err != nil { + t.Fatal(err) + } + if err := os.Symlink(target, link); err != nil { + t.Skipf("symlink capability unavailable: %v", err) + } + if err := Publish(context.Background(), link, data); !errors.Is(err, ErrWrite) { + t.Fatalf("symlink output: %v", err) + } + if _, err := Load(context.Background(), link); !errors.Is(err, ErrRead) { + t.Fatalf("symlink input: %v", err) + } + got, err := os.ReadFile(target) + if err != nil || !bytes.Equal(got, data) { + t.Fatal("symlink target changed") + } +} diff --git a/internal/bundle/file_unix.go b/internal/bundle/file_unix.go new file mode 100644 index 0000000..d0441cf --- /dev/null +++ b/internal/bundle/file_unix.go @@ -0,0 +1,8 @@ +//go:build !windows + +package bundle + +import "os" + +func privateFile(file *os.File) error { return file.Chmod(0600) } +func replaceFile(from, to string) error { return os.Rename(from, to) } diff --git a/internal/bundle/file_windows.go b/internal/bundle/file_windows.go new file mode 100644 index 0000000..ec0973e --- /dev/null +++ b/internal/bundle/file_windows.go @@ -0,0 +1,39 @@ +//go:build windows + +package bundle + +import ( + "os" + + "golang.org/x/sys/windows" +) + +func privateFile(file *os.File) error { + user, err := windows.GetCurrentProcessToken().GetTokenUser() + if err != nil { + return err + } + sd, err := windows.SecurityDescriptorFromString("D:P(A;;FA;;;SY)(A;;FA;;;" + user.User.Sid.String() + ")") + if err != nil { + return err + } + dacl, _, err := sd.DACL() + if err != nil { + return err + } + // os.CreateTemp's handle has data access, not WRITE_DAC. The file remains + // open without delete sharing while this API obtains the required access. + return windows.SetNamedSecurityInfo(file.Name(), windows.SE_FILE_OBJECT, windows.DACL_SECURITY_INFORMATION|windows.PROTECTED_DACL_SECURITY_INFORMATION, nil, nil, dacl, nil) +} + +func replaceFile(from, to string) error { + f, err := windows.UTF16PtrFromString(from) + if err != nil { + return err + } + t, err := windows.UTF16PtrFromString(to) + if err != nil { + return err + } + return windows.MoveFileEx(f, t, windows.MOVEFILE_REPLACE_EXISTING|windows.MOVEFILE_WRITE_THROUGH) +} diff --git a/internal/bundle/file_windows_test.go b/internal/bundle/file_windows_test.go new file mode 100644 index 0000000..73164f6 --- /dev/null +++ b/internal/bundle/file_windows_test.go @@ -0,0 +1,65 @@ +//go:build windows + +package bundle + +import ( + "bytes" + "context" + "errors" + "os" + "path/filepath" + "testing" + + "golang.org/x/sys/windows" +) + +func TestM4WindowsAtomicReplaceLockedDestination(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "capture.zip") + previous := []byte("previous regular destination") + if err := os.WriteFile(path, previous, 0600); err != nil { + t.Fatal(err) + } + p, err := windows.UTF16PtrFromString(path) + if err != nil { + t.Fatal(err) + } + h, err := windows.CreateFile(p, windows.GENERIC_READ, windows.FILE_SHARE_READ, nil, windows.OPEN_EXISTING, windows.FILE_ATTRIBUTE_NORMAL, 0) + if err != nil { + t.Fatal(err) + } + defer func() { _ = windows.CloseHandle(h) }() + if err := Publish(context.Background(), path, publicationArchive(t)); !errors.Is(err, ErrWrite) { + t.Fatalf("locked replacement: %v", err) + } + got, err := os.ReadFile(path) + if err != nil || !bytes.Equal(got, previous) { + t.Fatal("locked destination changed") + } + entries, err := os.ReadDir(dir) + if err != nil || len(entries) != 1 { + t.Fatal("locked replacement retained temporary file") + } +} + +func TestM4WindowsPublicationHasPrivateProtectedACL(t *testing.T) { + path := filepath.Join(t.TempDir(), "capture.zip") + if err := Publish(context.Background(), path, publicationArchive(t)); err != nil { + t.Fatal(err) + } + sd, err := windows.GetNamedSecurityInfo(path, windows.SE_FILE_OBJECT, windows.DACL_SECURITY_INFORMATION) + if err != nil { + t.Fatal(err) + } + control, _, err := sd.Control() + if err != nil { + t.Fatal(err) + } + if control&windows.SE_DACL_PROTECTED == 0 { + t.Fatal("staged publication inherited a non-private ACL") + } + dacl, _, err := sd.DACL() + if err != nil || dacl == nil { + t.Fatal("private DACL missing") + } +} diff --git a/internal/bundle/json.go b/internal/bundle/json.go new file mode 100644 index 0000000..68e8754 --- /dev/null +++ b/internal/bundle/json.go @@ -0,0 +1,196 @@ +package bundle + +import ( + "bytes" + "context" + "encoding/json" + "io" + "reflect" + "strconv" + "strings" + "sync" + "time" + "unicode/utf8" +) + +type jsonField struct { + typ reflect.Type + required bool +} + +var jsonFields sync.Map + +func fieldsFor(typ reflect.Type) map[string]jsonField { + if cached, ok := jsonFields.Load(typ); ok { + return cached.(map[string]jsonField) + } + fields := map[string]jsonField{} + for i := 0; i < typ.NumField(); i++ { + field := typ.Field(i) + tag := strings.Split(field.Tag.Get("json"), ",") + if tag[0] == "" || tag[0] == "-" { + continue + } + fields[tag[0]] = jsonField{typ: field.Type, required: !allowed("omitempty", tag[1:]...)} + } + jsonFields.Store(typ, fields) + return fields +} + +type jsonScanner struct { + ctx context.Context + decoder *json.Decoder + tokens int +} + +func (s *jsonScanner) token() (json.Token, error) { + if err := s.ctx.Err(); err != nil { + return nil, err + } + s.tokens++ + if s.tokens > 200_000 { + return nil, ErrBounds + } + t, err := s.decoder.Token() + if err != nil { + return nil, ErrInvalid + } + if value, ok := t.(string); ok && len(value) > 4096 { + return nil, ErrBounds + } + if value, ok := t.(json.Number); ok && len(value) > 32 { + return nil, ErrBounds + } + return t, nil +} + +func (s *jsonScanner) value(typ reflect.Type, depth int) error { + if depth > 32 { + return ErrBounds + } + t, err := s.token() + if err != nil { + return err + } + if typ.Kind() == reflect.Pointer { + // Only the required legacy identity presence markers are nullable. + if t == nil && typ.Elem().Kind() == reflect.String { + return nil + } + typ = typ.Elem() + } + if typ == reflect.TypeFor[time.Time]() { + value, ok := t.(string) + if !ok || !strings.HasSuffix(value, "Z") { + return ErrInvalid + } + parsed, err := time.Parse(time.RFC3339Nano, value) + if err != nil || parsed.IsZero() || parsed.Format(time.RFC3339Nano) != value { + return ErrInvalid + } + return nil + } + switch typ.Kind() { + case reflect.Struct: + if t != json.Delim('{') { + return ErrInvalid + } + fields, seen := fieldsFor(typ), map[string]bool{} + for s.decoder.More() { + key, err := s.token() + if err != nil { + return err + } + name, ok := key.(string) + if !ok || seen[name] { + return ErrInvalid + } + field, ok := fields[name] + if !ok { + return ErrInvalid + } + seen[name] = true + if err := s.value(field.typ, depth+1); err != nil { + return err + } + } + end, err := s.token() + if err != nil { + return err + } + if end != json.Delim('}') { + return ErrInvalid + } + for name, field := range fields { + if field.required && !seen[name] { + return ErrInvalid + } + } + case reflect.Slice: + if t != json.Delim('[') { + return ErrInvalid + } + count := 0 + for s.decoder.More() { + count++ + if count > 4096 { + return ErrBounds + } + if err := s.value(typ.Elem(), depth+1); err != nil { + return err + } + } + end, err := s.token() + if err != nil { + return err + } + if end != json.Delim(']') { + return ErrInvalid + } + case reflect.String: + if _, ok := t.(string); !ok { + return ErrInvalid + } + case reflect.Bool: + if _, ok := t.(bool); !ok { + return ErrInvalid + } + case reflect.Int, reflect.Int64: + number, ok := t.(json.Number) + if !ok { + return ErrInvalid + } + if _, err := strconv.ParseInt(string(number), 10, typ.Bits()); err != nil { + return ErrInvalid + } + default: + return ErrInvalid + } + return nil +} + +// Preflight grammar/work/required fields without constructing an interface{} +// graph. Exact JSON tags also reject encoding/json's case-insensitive aliases. +func decodeJSON(ctx context.Context, data []byte, destination any) error { + if !utf8.Valid(data) { + return ErrInvalid + } + decoder := json.NewDecoder(bytes.NewReader(data)) + decoder.UseNumber() + s := jsonScanner{ctx: ctx, decoder: decoder} + if err := s.value(reflect.TypeOf(destination).Elem(), 1); err != nil { + return err + } + if _, err := decoder.Token(); err != io.EOF { + return ErrInvalid + } + if err := ctx.Err(); err != nil { + return err + } + decoder = json.NewDecoder(bytes.NewReader(data)) + decoder.DisallowUnknownFields() + if err := decoder.Decode(destination); err != nil { + return ErrInvalid + } + return nil +} diff --git a/internal/bundle/model.go b/internal/bundle/model.go new file mode 100644 index 0000000..3dfb3b8 --- /dev/null +++ b/internal/bundle/model.go @@ -0,0 +1,198 @@ +// Package bundle implements the separate allowlisted sharing format in ADR 0006. +// Archives are data: no member is extracted, executed or used as a live target. +package bundle + +import ( + "errors" + "time" + + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/report" +) + +const ( + Version = 1 + Policy = "allowlist-v1" + MaxArchiveBytes = 2_000_000 + MaxUncompressedBytes = 8 << 20 + MaxEntries = 8 +) + +var ( + ErrInvalid = errors.New("invalid diagnostic bundle") + ErrBounds = errors.New("diagnostic bundle exceeds a resource limit") + ErrVersion = errors.New("unsupported diagnostic bundle version") + ErrIntegrity = errors.New("diagnostic bundle integrity check failed") + ErrEvidence = errors.New("diagnostic evidence cannot be represented by the sharing policy") + ErrRead = errors.New("cannot read diagnostic bundle") + ErrWrite = errors.New("cannot publish diagnostic bundle") +) + +var memberLimits = map[string]int{ + "manifest.json": 64 << 10, "snapshot.json": 2 << 20, + "report.json": 3 << 20, "report.md": 2 << 20, + "versions.json": 64 << 10, "diagnostics.json": 512 << 10, +} + +var requiredMembers = []string{"manifest.json", "snapshot.json", "report.json", "report.md", "versions.json"} + +type Options struct{ IncludeLogs bool } + +type Member struct { + Name string `json:"name"` + Size int `json:"size"` + SHA256 string `json:"sha256"` +} + +type Manifest struct { + BundleVersion int `json:"bundleVersion"` + SnapshotVersion int `json:"snapshotVersion"` + ReportVersion int `json:"reportVersion"` + RedactionPolicy string `json:"redactionPolicy"` + Tool report.Tool `json:"tool"` + CapturedAt time.Time `json:"capturedAt"` + CaptureExitCode int `json:"captureExitCode"` + Omissions []report.Omission `json:"omissions"` + Members []Member `json:"members"` +} + +// SharingSnapshot is deliberately separate from collector.Snapshot. New raw +// collector fields cannot start appearing in exports through struct copying. +type SharingSnapshot struct { + SchemaVersion int `json:"schemaVersion"` + CollectedAt time.Time `json:"collectedAt"` + Platform *Platform `json:"platform,omitempty"` + Shell *Shell `json:"shell,omitempty"` + Path *PathInfo `json:"path,omitempty"` + Runtimes []Runtime `json:"runtimes,omitempty"` + Ports []Port `json:"ports,omitempty"` + RequestedPorts []int `json:"requestedPorts,omitempty"` + Docker *Docker `json:"docker,omitempty"` + Git *Git `json:"git,omitempty"` + Dependencies *Dependencies `json:"dependencies,omitempty"` + PythonEnvironment *PythonEnvironment `json:"pythonEnvironment,omitempty"` + Proxies []Proxy `json:"proxies,omitempty"` + NoProxyConfigured bool `json:"noProxyConfigured,omitempty"` + Errors []Failure `json:"errors,omitempty"` +} + +type Platform struct { + OS string `json:"os"` + Version string `json:"version"` + Arch string `json:"arch"` + Elevated bool `json:"elevated"` +} +type Shell struct { + Name string `json:"name"` +} +type PathInfo struct { + Entries []PathEntry `json:"entries"` +} +type PathEntry struct { + Token string `json:"token"` + Exists bool `json:"exists"` + IsDir bool `json:"isDir"` + Position int `json:"position"` + Duplicate bool `json:"duplicate"` +} +type Runtime struct { + Name string `json:"name"` + Version string `json:"version"` + PathToken string `json:"pathToken"` + Manager string `json:"manager"` + Active bool `json:"active"` + Expected bool `json:"expected,omitempty"` + Missing bool `json:"missing,omitempty"` + ResolvedToken string `json:"resolvedToken,omitempty"` + Resolution string `json:"resolution,omitempty"` + EnvironmentToken string `json:"environmentToken,omitempty"` +} +type Port struct { + Port int `json:"port"` + Protocol string `json:"protocol"` + State string `json:"state"` +} +type Docker struct { + CLIVersion string `json:"cliVersion"` + Installed bool `json:"installed"` + DaemonReachable bool `json:"daemonReachable"` + ProbeStatus string `json:"probeStatus,omitempty"` + ClientAPI string `json:"clientApi,omitempty"` + ClientMinAPI string `json:"clientMinApi,omitempty"` + ClientDefaultAPI string `json:"clientDefaultApi,omitempty"` + ServerAPI string `json:"serverApi,omitempty"` + ServerMinAPI string `json:"serverMinApi,omitempty"` + APIOverride string `json:"apiOverride,omitempty"` +} +type Git struct { + Version string `json:"version"` + UserNamePresent bool `json:"userNamePresent"` + UserEmailPresent bool `json:"userEmailPresent"` + IdentityChecked bool `json:"identityChecked"` + Autocrlf string `json:"autocrlf"` + AutocrlfOriginKnown bool `json:"autocrlfOriginKnown"` +} +type Dependencies struct { + ProjectToken string `json:"projectToken"` + RequirementsDeclared bool `json:"requirementsDeclared"` + Requirements []Dependency `json:"requirements,omitempty"` + Package *PackageEvidence `json:"package,omitempty"` + GitAutocrlf string `json:"gitAutocrlf,omitempty"` +} +type Dependency struct { + Kind string `json:"kind"` + Name string `json:"name"` + PathToken string `json:"pathToken,omitempty"` + Status string `json:"status"` +} +type PackageManager struct { + Name string `json:"name"` + Version string `json:"version,omitempty"` +} +type PackageEvidence struct { + ManifestPresent bool `json:"manifestPresent"` + Declared *PackageManager `json:"declared,omitempty"` + Runner *PackageManager `json:"runner,omitempty"` + Lockfiles []string `json:"lockfiles,omitempty"` +} +type PythonEnvironment struct { + RootToken string `json:"rootToken"` + CanonicalToken string `json:"canonicalToken,omitempty"` + MetadataKnown bool `json:"metadataKnown"` +} +type Proxy struct { + Variable string `json:"variable"` + Scheme string `json:"scheme"` + HostToken string `json:"hostToken"` + Port int `json:"port"` + Probe string `json:"probe"` +} +type Failure struct { + Collector string `json:"collector"` + Reason string `json:"reason"` +} +type Diagnostic struct { + Collector string `json:"collector"` + DurationMs int64 `json:"durationMs"` + Status string `json:"status"` +} +type Diagnostics struct { + SchemaVersion int `json:"schemaVersion"` + Collectors []Diagnostic `json:"collectors"` +} +type RuntimeVersion struct { + Name string `json:"name"` + Version string `json:"version"` +} +type Versions struct { + SchemaVersion int `json:"schemaVersion"` + Tool report.Tool `json:"tool"` + Runtimes []RuntimeVersion `json:"runtimes"` + Git string `json:"git,omitempty"` + Docker string `json:"docker,omitempty"` +} + +type Capture struct { + Snapshot collector.Snapshot + Source report.Source +} diff --git a/internal/bundle/model_coverage.go b/internal/bundle/model_coverage.go new file mode 100644 index 0000000..8648d07 --- /dev/null +++ b/internal/bundle/model_coverage.go @@ -0,0 +1,47 @@ +package bundle + +import ( + "reflect" + "strings" + "sync" + + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/report" +) + +// A new source-model field requires an explicit sharing decision. Separate +// output types prevent raw copying; this guard also prevents silently ignoring +// a future fact that might be a new prerequisite. Order is intentionally exact. +var currentSourceModel = sync.OnceValue(func() bool { + for typ, names := range map[reflect.Type]string{ + reflect.TypeFor[collector.Snapshot](): "SchemaVersion CollectedAt Platform Shell Path Runtimes Ports RequestedPorts Docker Git Dependencies PythonEnvironment Proxies NoProxyConfigured Errors", + reflect.TypeFor[collector.Platform](): "OS Version Arch Elevated", + reflect.TypeFor[collector.Shell](): "Name Path", + reflect.TypeFor[collector.PathInfo](): "Entries", + reflect.TypeFor[collector.PathEntry](): "Raw Exists IsDir Position Duplicate", + reflect.TypeFor[collector.Runtime](): "Name Version Path Manager Active Expected Missing ResolvedPath Resolution EnvironmentRoot", + reflect.TypeFor[collector.Port](): "Port Protocol State PID Process", + reflect.TypeFor[collector.Docker](): "CLIVersion DaemonReachable Context Installed ProbeStatus ClientAPI ClientMinAPI ClientDefaultAPI ServerAPI ServerMinAPI APIOverride SelectedContext SelectedHost", + reflect.TypeFor[collector.Git](): "Version UserName UserEmail Autocrlf IdentityChecked UserNameOrigin UserEmailOrigin AutocrlfOrigin", + reflect.TypeFor[collector.Dependencies](): "ProjectRoot RequirementsFile Requirements Package GitAutocrlf", + reflect.TypeFor[collector.Dependency](): "Kind Name Path Status", + reflect.TypeFor[collector.PackageManager](): "Name Version", + reflect.TypeFor[collector.PackageEvidence](): "Manifest Declared Runner Lockfiles", + reflect.TypeFor[collector.PythonEnvironment](): "Root CanonicalRoot MetadataKnown", + reflect.TypeFor[collector.Proxy](): "Variable URL Host Port Probe", + reflect.TypeFor[collector.CollectorError](): "Collector Message", + reflect.TypeFor[report.Tool](): "Name Version", + reflect.TypeFor[report.Collector](): "Name DurationMs Error", + } { + fields := strings.Fields(names) + if typ.NumField() != len(fields) { + return false + } + for i, name := range fields { + if typ.Field(i).Name != name { + return false + } + } + } + return true +}) diff --git a/internal/bundle/parser_test.go b/internal/bundle/parser_test.go new file mode 100644 index 0000000..2c706fd --- /dev/null +++ b/internal/bundle/parser_test.go @@ -0,0 +1,65 @@ +package bundle + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "io" + "strings" + "testing" +) + +type deepJSON struct { + Child *deepJSON `json:"child,omitempty"` +} +type tokenJSON struct { + Rows [][]int `json:"rows"` +} +type stringJSON struct { + Value string `json:"value"` +} + +func TestM4JSONIndependentWorkBounds(t *testing.T) { + for _, tc := range []struct { + name string + data []byte + target any + want error + }{ + {"depth", []byte(strings.Repeat(`{"child":`, 32) + `{}` + strings.Repeat(`}`, 32)), &deepJSON{}, ErrBounds}, + {"array", []byte(`{"rows":[[` + strings.Repeat(`1,`, 4096) + `1]]}`), &tokenJSON{}, ErrBounds}, + {"tokens", []byte(`{"rows":[` + strings.Repeat(`[`+strings.Repeat(`1,`, 4095)+`1],`, 50) + `[` + strings.Repeat(`1,`, 4095) + `1]]}`), &tokenJSON{}, ErrBounds}, + {"UTF8-byte-length", []byte(`{"value":"` + strings.Repeat("界", 1366) + `"}`), &stringJSON{}, ErrBounds}, + {"escaped-key-duplicate", []byte(`{"value":"one","\u0076alue":"two"}`), &stringJSON{}, ErrInvalid}, + {"case-alias", []byte(`{"Value":"one"}`), &stringJSON{}, ErrInvalid}, + {"null-required", []byte(`{"value":null}`), &stringJSON{}, ErrInvalid}, + {"missing-required", []byte(`{}`), &stringJSON{}, ErrInvalid}, + } { + t.Run(tc.name, func(t *testing.T) { + if err := decodeJSON(context.Background(), tc.data, tc.target); !errors.Is(err, tc.want) { + t.Fatalf("got=%v want=%v", err, tc.want) + } + }) + } + data, _ := json.Marshal(stringJSON{Value: strings.Repeat("A", 4096)}) + if err := decodeJSON(context.Background(), data, &stringJSON{}); err != nil { + t.Fatalf("exact string boundary: %v", err) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if err := decodeJSON(ctx, data, &stringJSON{}); !errors.Is(err, context.Canceled) { + t.Fatalf("cancel=%v", err) + } +} + +func TestM4ArchiveBufferCannotBypassSizeGuard(t *testing.T) { + out := &archiveBuffer{ctx: context.Background()} + if _, ok := any(out).(io.ReaderFrom); ok { + t.Fatal("unbounded ReaderFrom exposed") + } + _, err := io.Copy(out, bytes.NewReader(bytes.Repeat([]byte{0}, MaxArchiveBytes))) + if !errors.Is(err, ErrBounds) || out.Len() >= MaxArchiveBytes { + t.Fatalf("guard bypassed: size=%d error=%v", out.Len(), err) + } +} diff --git a/internal/bundle/roundtrip_test.go b/internal/bundle/roundtrip_test.go new file mode 100644 index 0000000..93ed9af --- /dev/null +++ b/internal/bundle/roundtrip_test.go @@ -0,0 +1,213 @@ +package bundle_test + +import ( + "bytes" + "context" + "encoding/json" + "os" + "path/filepath" + "reflect" + "runtime" + "strings" + "testing" + "time" + + "github.com/shellghostt/envdoctor/internal/bundle" + "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" +) + +func TestM4All172GoldenCapturesRoundTripAcrossOS(t *testing.T) { + dir := filepath.Join("..", "..", "tests", "golden") + files, err := filepath.Glob(filepath.Join(dir, "*.json")) + if err != nil { + t.Fatal(err) + } + if len(files) != 172 { + t.Fatalf("original golden corpus changed: %d", len(files)) + } + osSeen := map[string]bool{} + for _, file := range files { + t.Run(filepath.Base(file), func(t *testing.T) { + raw, err := os.ReadFile(file) + if err != nil { + t.Fatal(err) + } + fixture, err := diagnostic.LoadFixture(bytes.NewReader(raw)) + if err != nil { + t.Fatal(err) + } + s := fixture.Snapshot + if s.Platform != nil { + osSeen[s.Platform.OS] = true + } + findings, err := diagnostic.DefaultEngine().Evaluate(s, diagnostic.Filter{}) + if err != nil { + t.Fatal(err) + } + doc := report.NewDocument("0.1.0", s.CollectedAt, s, findings, nil, 12) + data, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil { + t.Fatal(err) + } + capture, err := bundle.Read(context.Background(), data) + if err != nil { + t.Fatal(err) + } + got, err := diagnostic.DefaultEngine().Evaluate(capture.Snapshot, diagnostic.Filter{RuleID: fixture.RuleID}) + if err != nil { + t.Fatal(err) + } + metadata, _ := diagnostic.LookupRule(fixture.RuleID) + blocked := false + for _, failure := range capture.Snapshot.Errors { + for _, pre := range metadata.Prerequisites { + blocked = blocked || failure.Collector == pre + } + } + if blocked { + if len(got) != 0 { + t.Fatal("unavailable prerequisite produced conclusive findings") + } + return + } + if len(got) != len(fixture.ExpectedFindings) { + t.Fatalf("findings lost: got=%v expected=%v", got, fixture.ExpectedFindings) + } + for i, f := range got { + want := fixture.ExpectedFindings[i] + // Evidence locations/owners intentionally change; verdict and all + // registry-owned policy/advice must remain identical. + f.Detected = nil + want.Detected = nil + if !reflect.DeepEqual(f, want) { + t.Fatalf("retained verdict changed: got=%+v want=%+v", f, want) + } + } + again, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil || !bytes.Equal(data, again) { + t.Fatal("same capture did not produce deterministic aliases/archive") + } + }) + } + for _, goos := range []string{"windows", "linux", "darwin"} { + if !osSeen[goos] { + t.Errorf("capture OS %s missing", goos) + } + } +} + +func TestM4EnvironmentAliasUsesCapturedOSAndExactPaths(t *testing.T) { + for _, goos := range []string{"windows", "linux", "darwin"} { + doc := sensitiveDocument() + doc.Snapshot.Errors = nil + doc.Snapshot.Platform.OS = goos + doc.Snapshot.Runtimes = doc.Snapshot.Runtimes[:1] + doc.Snapshot.PythonEnvironment.CanonicalRoot = "/Capture/ENV" + doc.Snapshot.PythonEnvironment.Root = "/Capture/ENV" + doc.Snapshot.Runtimes[0].EnvironmentRoot = "/capture/env/./" + data, err := bundle.Build(context.Background(), doc, bundle.Options{}) + if err != nil { + t.Fatal(err) + } + capture, err := bundle.Read(context.Background(), data) + if err != nil { + t.Fatal(err) + } + results, err := diagnostic.DefaultEngine().Evaluate(capture.Snapshot, diagnostic.Filter{RuleID: "ENV011"}) + if err != nil { + t.Fatal(err) + } + if (len(results) > 0) != (goos != "windows") { + t.Fatalf("capture %s used receiving OS normalization", goos) + } + } + doc := sensitiveDocument() + doc.Snapshot.Errors = nil + doc.Snapshot.Runtimes[1].Path = strings.ToLower(doc.Snapshot.Runtimes[0].Path) + data, err := bundle.Build(context.Background(), doc, bundle.Options{}) + if err != nil { + t.Fatal(err) + } + capture, err := bundle.Read(context.Background(), data) + if err != nil { + t.Fatal(err) + } + if capture.Snapshot.Runtimes[0].Path == capture.Snapshot.Runtimes[1].Path { + t.Fatal("exact path comparison collapsed distinct case") + } +} + +func TestM4RepeatedBundleLifecycleReleasesResources(t *testing.T) { + doc := sensitiveDocument() + data, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil { + t.Fatal(err) + } + runtime.GC() + var before, after runtime.MemStats + runtime.ReadMemStats(&before) + beforeG := runtime.NumGoroutine() + start := time.Now() + for i := 0; i < 100; i++ { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + out, err := bundle.Build(ctx, doc, bundle.Options{IncludeLogs: true}) + if err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(ctx, out); err != nil { + t.Fatal(err) + } + cancel() + if _, err := bundle.Read(ctx, out); err == nil { + t.Fatal("canceled read succeeded") + } + } + runtime.GC() + runtime.ReadMemStats(&after) + afterG := runtime.NumGoroutine() + t.Logf("100 bundle build/read cycles: %s; archive %d bytes; goroutines %d -> %d; retained heap %d -> %d", time.Since(start), len(data), beforeG, afterG, before.HeapAlloc, after.HeapAlloc) + if afterG > beforeG+2 || int64(after.HeapAlloc)-int64(before.HeapAlloc) > 2<<20 { + t.Fatal("bundle resources retained") + } +} + +func BenchmarkM4BundleBuildRead(b *testing.B) { + doc := sensitiveDocument() + data, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil { + b.Fatal(err) + } + b.SetBytes(int64(len(data))) + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + data, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil { + b.Fatal(err) + } + if _, err := bundle.Read(context.Background(), data); err != nil { + b.Fatal(err) + } + } +} + +func FuzzM4BundleReader(f *testing.F) { + data, err := bundle.Build(context.Background(), sensitiveDocument(), bundle.Options{IncludeLogs: true}) + if err != nil { + f.Fatal(err) + } + f.Add(data) + f.Add([]byte("not a zip")) + f.Add([]byte{}) + f.Fuzz(func(t *testing.T, data []byte) { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + capture, err := bundle.Read(ctx, data) + if err == nil { + if _, err := json.Marshal(capture); err != nil { + t.Fatal(err) + } + } + }) +} diff --git a/internal/bundle/schema_test.go b/internal/bundle/schema_test.go new file mode 100644 index 0000000..2325a79 --- /dev/null +++ b/internal/bundle/schema_test.go @@ -0,0 +1,129 @@ +package bundle_test + +import ( + "archive/zip" + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "testing" + + "github.com/santhosh-tekuri/jsonschema/v5" + "github.com/shellghostt/envdoctor/internal/bundle" + "github.com/shellghostt/envdoctor/internal/report" +) + +func compileBundleSchemas(t *testing.T) map[string]*jsonschema.Schema { + t.Helper() + compiler := jsonschema.NewCompiler() + compiler.LoadURL = func(string) (io.ReadCloser, error) { + return nil, fmt.Errorf("unregistered schema; network loading forbidden") + } + dir := filepath.Join("..", "..", "docs", "schema") + files, err := filepath.Glob(filepath.Join(dir, "*.schema.json")) + if err != nil { + t.Fatal(err) + } + for _, file := range files { + data, err := os.ReadFile(file) + if err != nil { + t.Fatal(err) + } + var id struct { + ID string `json:"$id"` + } + if err := json.Unmarshal(data, &id); err != nil { + t.Fatal(err) + } + if err := compiler.AddResource(id.ID, bytes.NewReader(data)); err != nil { + t.Fatal(err) + } + } + result := map[string]*jsonschema.Schema{} + for member, name := range map[string]string{"manifest.json": "bundle-manifest-v1", "snapshot.json": "bundle-snapshot-v1", "report.json": "bundle-report-v1", "versions.json": "bundle-versions-v1", "diagnostics.json": "bundle-diagnostics-v1"} { + schema, err := compiler.Compile("https://github.com/shellghostt/envdoctor/schema/" + name + ".schema.json") + if err != nil { + t.Fatal(err) + } + result[member] = schema + } + return result +} + +func TestM4BundleMembersMatchCommittedSchemas(t *testing.T) { + schemas := compileBundleSchemas(t) + unsupported := sensitiveDocument() + unsupported.Snapshot.Runtimes[0].Version = "SECRET-VERSION" + for _, doc := range []report.Document{sensitiveDocument(), unsupported} { + data, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil { + t.Fatal(err) + } + for name, body := range members(t, data) { + if name == "report.md" { + continue + } + var value any + if err := json.Unmarshal(body, &value); err != nil { + t.Fatal(err) + } + if err := schemas[name].Validate(value); err != nil { + t.Fatalf("%s does not match committed schema: %v", name, err) + } + } + } +} + +func TestM4OptionalEmptyAliasesMatchSchemaAndReader(t *testing.T) { + doc := sensitiveDocument() + for i := range doc.Snapshot.Runtimes { + doc.Snapshot.Runtimes[i].ResolvedPath = "" + doc.Snapshot.Runtimes[i].EnvironmentRoot = "" + } + doc.Snapshot.Dependencies.Requirements[0].Path = "" + doc.Snapshot.PythonEnvironment.CanonicalRoot = "" + doc.Snapshot.PythonEnvironment.MetadataKnown = false + data, err := bundle.Build(context.Background(), doc, bundle.Options{}) + if err != nil { + t.Fatal(err) + } + all := members(t, data) + var snapshot map[string]any + if err := json.Unmarshal(all["snapshot.json"], &snapshot); err != nil { + t.Fatal(err) + } + for _, value := range snapshot["runtimes"].([]any) { + runtime := value.(map[string]any) + runtime["resolvedToken"], runtime["environmentToken"] = "", "" + } + requirement := snapshot["dependencies"].(map[string]any)["requirements"].([]any)[0].(map[string]any) + requirement["pathToken"] = "" + snapshot["pythonEnvironment"].(map[string]any)["canonicalToken"] = "" + schema := compileBundleSchemas(t)["snapshot.json"] + if err := schema.Validate(snapshot); err != nil { + t.Fatalf("empty optional aliases disagree with schema: %v", err) + } + all["snapshot.json"], err = json.Marshal(snapshot) + if err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(context.Background(), repack(t, all, zip.Deflate)); err != nil { + t.Fatalf("empty optional aliases disagree with reader: %v", err) + } + for _, invalid := range []string{"C:/Users/SECRET-IDENTITY", "path-0000", "path-4097"} { + requirement["pathToken"] = invalid + if err := schema.Validate(snapshot); err == nil { + t.Fatalf("schema accepted invalid alias %q", invalid) + } + all["snapshot.json"], err = json.Marshal(snapshot) + if err != nil { + t.Fatal(err) + } + if _, err := bundle.Read(context.Background(), repack(t, all, zip.Deflate)); err == nil { + t.Fatalf("reader accepted invalid alias %q", invalid) + } + } +} diff --git a/internal/bundle/validate.go b/internal/bundle/validate.go new file mode 100644 index 0000000..2b1b6da --- /dev/null +++ b/internal/bundle/validate.go @@ -0,0 +1,274 @@ +package bundle + +import ( + "crypto/sha256" + "encoding/hex" + "reflect" + "strconv" + "strings" + + "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" +) + +var omissionFields = []string{"platform", "platform.version", "shell.name", "shell.path", "path.locations", "runtimes.paths", "runtimes.version", "runtimes.name", "runtimes.manager", "runtimes.resolution", "runtimes.environment", "network.ownership", "docker.context", "docker.endpoint", "docker.version", "docker.api", "git.identity", "git.origins", "git.version", "git.autocrlf", "dependencies.locations", "dependencies.requirement", "dependencies.packageManager", "dependencies.lockfile", "dependencies.gitAutocrlf", "pythonEnvironment.locations", "proxy.endpoint", "proxy.url", "collector.errors", "tool.version"} + +func validToken(value string, kind string, optional bool) bool { + if value == "" { + return optional + } + var digits string + switch kind { + case "path": + if !pathToken.MatchString(value) { + return false + } + digits = value[5:] + case "env": + if !environmentToken.MatchString(value) { + return false + } + digits = value[4:] + case "host": + if !hostToken.MatchString(value) { + return false + } + digits = strings.TrimSuffix(value[5:], ".invalid") + default: + return false + } + n, err := strconv.Atoi(digits) + return err == nil && n >= 1 && n <= 4096 +} + +func validateSnapshot(s SharingSnapshot) error { + if s.SchemaVersion != 1 { + return ErrVersion + } + if s.CollectedAt.IsZero() { + return ErrInvalid + } + if len(s.Runtimes) > 128 || len(s.Ports) > 1024 || len(s.RequestedPorts) > 1024 || len(s.Proxies) > 3 || len(s.Errors) > 32 || s.Path != nil && len(s.Path.Entries) > 1024 || s.Dependencies != nil && len(s.Dependencies.Requirements) > 32 { + return ErrBounds + } + if p := s.Platform; p != nil { + if !allowed(p.OS, "windows", "linux", "darwin") || !allowed(p.Arch, "amd64", "arm64", "386", "arm", "ppc64le", "s390x", "riscv64") || !version(p.Version) { + return ErrInvalid + } + } + if s.Shell != nil && !knownShell(s.Shell.Name) { + return ErrInvalid + } + if p := s.Path; p != nil { + for i, e := range p.Entries { + if !validToken(e.Token, "path", false) || e.Position != i || e.IsDir && !e.Exists { + return ErrInvalid + } + } + } + for _, r := range s.Runtimes { + if !allowed(r.Name, "python", "node") || !knownManager(r.Manager) || !version(r.Version) || !validToken(r.PathToken, "path", false) || !validToken(r.ResolvedToken, "path", true) || !validToken(r.EnvironmentToken, "env", true) || !allowed(r.Resolution, "", "known", "unknown") { + return ErrInvalid + } + } + for _, p := range s.RequestedPorts { + if p < 1 || p > 65535 { + return ErrInvalid + } + } + for _, p := range s.Ports { + if p.Port < 1 || p.Port > 65535 || !allowed(p.Protocol, "tcp", "udp") || !allowed(p.State, "listen", "established", "closed", "unknown") { + return ErrInvalid + } + } + if d := s.Docker; d != nil { + if !version(d.CLIVersion) || !allowed(d.ProbeStatus, "", "absent", "available", "unavailable", "unknown", "denied", "unsupported") { + return ErrInvalid + } + for _, v := range []string{d.ClientAPI, d.ClientMinAPI, d.ClientDefaultAPI, d.ServerAPI, d.ServerMinAPI, d.APIOverride} { + if !knownAPI(v) { + return ErrInvalid + } + } + if d.ProbeStatus == "available" && (!d.Installed || !d.DaemonReachable) || d.ProbeStatus == "absent" && d.Installed { + return ErrInvalid + } + } + if g := s.Git; g != nil { + if !version(g.Version) || !knownAutocrlf(g.Autocrlf) { + return ErrInvalid + } + } + if d := s.Dependencies; d != nil { + if !validToken(d.ProjectToken, "path", true) || !knownAutocrlf(d.GitAutocrlf) { + return ErrInvalid + } + for _, r := range d.Requirements { + if !knownDependency(r.Kind, r.Name) || !allowed(r.Status, "present", "missing", "unknown") || !validToken(r.PathToken, "path", true) { + return ErrInvalid + } + } + if p := d.Package; p != nil { + for _, m := range []*PackageManager{p.Declared, p.Runner} { + if m != nil && (!knownPackageManager(m.Name) || m.Version != "" && !exactVersion(m.Version)) { + return ErrInvalid + } + } + if p.Declared != nil && !exactVersion(p.Declared.Version) { + return ErrInvalid + } + for _, lock := range p.Lockfiles { + if !knownLockfile(lock) { + return ErrInvalid + } + } + } + } + if e := s.PythonEnvironment; e != nil { + if !validToken(e.RootToken, "env", true) || !validToken(e.CanonicalToken, "env", true) || e.MetadataKnown && e.CanonicalToken == "" { + return ErrInvalid + } + } + seen := map[string]bool{} + for _, p := range s.Proxies { + if seen[p.Variable] || !allowed(p.Variable, "HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY") || !allowed(p.Scheme, "http", "https", "socks5", "socks5h") || !validToken(p.HostToken, "host", false) || p.Port < 1 || p.Port > 65535 || !allowed(p.Probe, "untested", "reachable", "unreachable", "unknown", "invalid") { + return ErrInvalid + } + seen[p.Variable] = true + } + seen = map[string]bool{} + for _, f := range s.Errors { + if !knownCollector(f.Collector) || f.Reason != "unavailable" || seen[f.Collector] { + return ErrInvalid + } + seen[f.Collector] = true + } + return nil +} + +func validateManifest(m Manifest, all map[string][]byte) error { + if m.BundleVersion != 1 || m.SnapshotVersion != 1 || m.ReportVersion != 1 || m.RedactionPolicy != Policy { + return ErrVersion + } + if m.Tool.Name != "envdoctor" || m.Tool.Version == "" || m.Tool.Version != "dev" && !numericVersion.MatchString(m.Tool.Version) || m.CapturedAt.IsZero() || m.CaptureExitCode < 0 || m.CaptureExitCode > 3 { + return ErrInvalid + } + if len(m.Omissions) > 256 || len(m.Members) > MaxEntries-1 { + return ErrBounds + } + seenOmissions := map[report.Omission]bool{} + for _, o := range m.Omissions { + if !allowed(o.Field, omissionFields...) || !allowed(o.Reason, "identifying_values", "raw_error", "unsupported_value", "unavailable_evidence") || seenOmissions[o] { + return ErrInvalid + } + seenOmissions[o] = true + } + if len(m.Members) != len(all)-1 { + return ErrInvalid + } + seen := map[string]bool{} + for _, member := range m.Members { + body, ok := all[member.Name] + if !ok || member.Name == "manifest.json" || seen[member.Name] || member.Size < 0 || member.Size != len(body) { + return ErrIntegrity + } + seen[member.Name] = true + digest := sha256.Sum256(body) + if member.SHA256 != hex.EncodeToString(digest[:]) { + return ErrIntegrity + } + } + return nil +} + +func makeVersions(s SharingSnapshot, tool report.Tool) Versions { + v := Versions{SchemaVersion: 1, Tool: tool, Runtimes: []RuntimeVersion{}} + seen := map[RuntimeVersion]bool{} + for _, r := range s.Runtimes { + item := RuntimeVersion{Name: r.Name, Version: r.Version} + if r.Version != "" && !seen[item] { + v.Runtimes = append(v.Runtimes, item) + seen[item] = true + } + } + if s.Git != nil { + v.Git = s.Git.Version + } + if s.Docker != nil { + v.Docker = s.Docker.CLIVersion + } + return v +} + +func sourceFor(m Manifest) report.Source { + return report.Source{Kind: "bundle", CaptureTool: m.Tool, CapturedAt: m.CapturedAt, RedactionPolicy: m.RedactionPolicy, CaptureExitCode: m.CaptureExitCode, Omissions: m.Omissions} +} + +func validateReference(doc report.Document, s SharingSnapshot, m Manifest, v Versions, diags *Diagnostics) error { + if doc.SchemaVersion != 1 || v.SchemaVersion != 1 { + return ErrVersion + } + expectedSource := sourceFor(m) + if doc.Source == nil || !reflect.DeepEqual(*doc.Source, expectedSource) || doc.Tool != m.Tool || !doc.GeneratedAt.Equal(m.CapturedAt) || !s.CollectedAt.Equal(m.CapturedAt) || !reflect.DeepEqual(doc.Snapshot, engineSnapshot(s)) || !reflect.DeepEqual(v, makeVersions(s, m.Tool)) { + return ErrInvalid + } + if doc.ExitCode < 0 || doc.ExitCode > 3 || doc.Rules < 0 || doc.Rules > 4096 { + return ErrInvalid + } + summary := report.Summary{Total: len(doc.Findings)} + for _, f := range doc.Findings { + if err := f.Validate(); err != nil { + return ErrInvalid + } + switch f.Severity { + case diagnostic.SeverityInfo: + summary.Info++ + case diagnostic.SeverityWarning: + summary.Warning++ + case diagnostic.SeverityError: + summary.Error++ + case diagnostic.SeverityCritical: + summary.Critical++ + } + } + if summary != doc.Summary { + return ErrInvalid + } + failed := map[string]bool{} + for _, f := range s.Errors { + failed[f.Collector] = true + } + seen := map[string]bool{} + for _, c := range doc.Collectors { + if !knownCollector(c.Name) || seen[c.Name] || c.DurationMs < 0 || c.DurationMs > maxDurationMs || c.Error != "" && c.Error != unavailableNotice || (c.Error != "") != failed[c.Name] { + return ErrInvalid + } + seen[c.Name] = true + } + for name := range failed { + if !seen[name] { + return ErrInvalid + } + } + if diags != nil { + if diags.SchemaVersion != 1 { + return ErrVersion + } + if len(diags.Collectors) > 32 { + return ErrBounds + } + seen = map[string]bool{} + for _, c := range diags.Collectors { + if !knownCollector(c.Collector) || seen[c.Collector] || c.DurationMs < 0 || c.DurationMs > maxDurationMs || !allowed(c.Status, "available", "unavailable") || (c.Status == "unavailable") != failed[c.Collector] { + return ErrInvalid + } + seen[c.Collector] = true + } + for name := range failed { + if !seen[name] { + return ErrInvalid + } + } + } + return nil +} diff --git a/internal/cli/bundle.go b/internal/cli/bundle.go new file mode 100644 index 0000000..5ce32aa --- /dev/null +++ b/internal/cli/bundle.go @@ -0,0 +1,61 @@ +package cli + +import ( + "fmt" + "io" + + "github.com/spf13/cobra" + + "github.com/shellghostt/envdoctor/internal/bundle" + "github.com/shellghostt/envdoctor/internal/config" +) + +func (a *App) newBundleCommand() *cobra.Command { + opts := config.Options{} + output := "envdoctor-bundle.zip" + includeLogs := false + cmd := &cobra.Command{ + Use: "bundle", Short: "Publish a redacted sharing bundle of the complete builtin diagnosis", + Long: "Capture approved diagnostic facts and publish a bounded ZIP atomically.\n" + + "Git identity becomes presence facts; paths and hosts become capture-local aliases.\n" + + "--include-logs includes only structured collector statuses/timings, never raw logs.\n" + + "Publication success exits 0; the capture's diagnosis exit is reference metadata.", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + if err := opts.Validate(); err != nil { + return err + } + if output == "" { + return fmt.Errorf("--output requires a file") + } + doc, err := a.captureDocument(cmd.Context(), opts, false) + if err != nil { + return toolFailure{err} + } + data, err := bundle.Build(cmd.Context(), doc, bundle.Options{IncludeLogs: includeLogs}) + if err != nil { + return toolFailure{err} + } + // Report bounded progress before committing the destination. A broken + // status stream must not turn a successful rename into a failed command + // that has already replaced the user's previous capture. + status := fmt.Sprintf("Writing diagnostic bundle (%d bytes); original diagnosis exit %d.\n", len(data), doc.ExitCode) + n, err := io.WriteString(a.Stdout, status) + if err == nil && n != len(status) { + err = io.ErrShortWrite + } + if err != nil { + return toolFailure{err} + } + if err := bundle.Publish(cmd.Context(), output, data); err != nil { + return toolFailure{err} + } + a.exitCode = ExitOK + return nil + }, + } + cmd.Flags().StringVarP(&output, "output", "o", output, "destination ZIP file") + cmd.Flags().BoolVar(&includeLogs, "include-logs", false, "include only allowlisted structured collector statuses/timings") + bindCaptureFlags(cmd, &opts) + return cmd +} diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 072bfd5..9329d7d 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -6,16 +6,21 @@ package cli import ( + "context" "errors" "fmt" "io" "os" + "strings" "time" "github.com/spf13/cobra" + "golang.org/x/term" + "github.com/shellghostt/envdoctor/internal/bundle" "github.com/shellghostt/envdoctor/internal/collector" "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" ) // Process exit codes. PLAN section 8 is the contract. @@ -29,16 +34,22 @@ const ( // App is one invocation of the binary. type App struct { - Version string - Stdout io.Writer - Stderr io.Writer - Getenv func(string) string - Now func() time.Time - Collectors []collector.Collector - HomeDir func() (string, error) - Engine *diagnostic.Engine - - exitCode int + Version string + Stdin io.Reader + Stdout io.Writer + Stderr io.Writer + Getenv func(string) string + LookupEnv func(string) (string, bool) + Context context.Context + Terminals func() report.TerminalState + Interactive func(context.Context, io.Reader, io.Writer, report.Document) error + Now func() time.Time + Collectors []collector.Collector + HomeDir func() (string, error) + Engine *diagnostic.Engine + + exitCode int + bundleMode bool } // toolFailure marks an error as a failure of the tool itself rather than of the @@ -52,15 +63,36 @@ func (e toolFailure) Unwrap() error { return e.err } // which keeps it usable from tests. func Main(args []string, app App) int { app.applyDefaults() + for _, arg := range args[1:] { + if arg == "bundle" || arg == "--from-bundle" || strings.HasPrefix(arg, "--from-bundle=") { + app.bundleMode = true + } + } root := app.newRoot() root.SetArgs(args[1:]) - if err := root.Execute(); err != nil { - home, _ := app.HomeDir() - _, _ = fmt.Fprintf(app.Stderr, "error: %s\n", collector.RedactText(err.Error(), home)) + if err := root.ExecuteContext(app.Context); err != nil { var failure toolFailure - if errors.As(err, &failure) { + failed := errors.As(err, &failure) + message := err.Error() + if app.bundleMode { + message = "invalid bundle command options" + if failed { + message = "diagnostic bundle operation failed" + for _, reason := range []error{bundle.ErrInvalid, bundle.ErrBounds, bundle.ErrVersion, bundle.ErrIntegrity, bundle.ErrEvidence, bundle.ErrRead, bundle.ErrWrite, context.Canceled, context.DeadlineExceeded} { + if errors.Is(err, reason) { + message = reason.Error() + break + } + } + } + } else { + home, _ := app.HomeDir() + message = collector.RedactText(message, home) + } + _, _ = fmt.Fprintf(app.Stderr, "error: %s\n", report.VisibleText(message)) + if failed { return ExitToolFailure } _, _ = fmt.Fprintln(app.Stderr, "Run 'envdoctor --help' for usage.") @@ -70,6 +102,9 @@ func Main(args []string, app App) int { } func (a *App) applyDefaults() { + if a.Context == nil { + a.Context = context.Background() + } if a.Stdout == nil { a.Stdout = io.Discard } @@ -79,6 +114,12 @@ func (a *App) applyDefaults() { if a.Getenv == nil { a.Getenv = os.Getenv } + if a.LookupEnv == nil { + a.LookupEnv = os.LookupEnv + } + if a.Interactive == nil { + a.Interactive = report.WriteInteractive + } if a.Now == nil { a.Now = time.Now } @@ -121,14 +162,29 @@ func (a *App) newRoot() *cobra.Command { } root.SetOut(a.Stdout) root.SetErr(a.Stderr) + root.SetIn(a.Stdin) root.AddCommand(a.newDiagnoseCommand()) + root.AddCommand(a.newBundleCommand()) root.AddCommand(a.newVersionCommand()) root.AddCommand(a.newExplainCommand()) root.AddCommand(a.newRulesCommand()) return root } +func terminalStream(stream any) bool { + file, ok := stream.(interface{ Fd() uintptr }) + return ok && term.IsTerminal(int(file.Fd())) +} + +func (a *App) terminalState() report.TerminalState { + if a.Terminals != nil { + return a.Terminals() + } + _, noColor := a.LookupEnv("NO_COLOR") + return report.TerminalState{Input: terminalStream(a.Stdin), Output: terminalStream(a.Stdout), Error: terminalStream(a.Stderr), Term: a.Getenv("TERM"), NoColor: noColor} +} + func (a *App) newVersionCommand() *cobra.Command { return &cobra.Command{ Use: "version", diff --git a/internal/cli/diagnose.go b/internal/cli/diagnose.go index 65bb575..938ad13 100644 --- a/internal/cli/diagnose.go +++ b/internal/cli/diagnose.go @@ -7,6 +7,7 @@ import ( "github.com/spf13/cobra" + "github.com/shellghostt/envdoctor/internal/bundle" "github.com/shellghostt/envdoctor/internal/collector" "github.com/shellghostt/envdoctor/internal/config" "github.com/shellghostt/envdoctor/internal/diagnostic" @@ -15,56 +16,104 @@ import ( func (a *App) newDiagnoseCommand() *cobra.Command { opts := config.Options{} - + format := report.FormatOptions{} cmd := &cobra.Command{ - Use: "diagnose", - Short: "Collect environment facts and report findings", - Long: "Collects facts about this machine and reports findings from the rules in this build.\n" + - "With --json, writes exactly one JSON document to stdout and nothing else.", + Use: "diagnose", Short: "Collect environment facts and report findings", + Long: "Report environment facts and manual advice as text, Markdown or JSON.\n" + + "Capable terminals use a scrollable viewer by default; --plain forces text.\n" + + "--from-bundle evaluates captured facts offline without inspecting this machine.", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, _ []string) error { + format.JSON, format.Explicit = opts.JSON, cmd.Flags().Changed("format") if err := opts.Validate(); err != nil { return err } + if cmd.Flags().Changed("from-bundle") { + if opts.FromBundle == "" { + return fmt.Errorf("--from-bundle requires a file") + } + for _, flag := range []string{"fixture-dir", "port", "probe-network", "package-manager"} { + if cmd.Flags().Changed(flag) { + return fmt.Errorf("--from-bundle conflicts with live capture flags and --fixture-dir") + } + } + } + // Validate formats before terminal lookup or evidence I/O. + mode, err := report.SelectMode(format, report.TerminalState{}) + if err != nil { + return err + } + if !format.JSON && !format.Explicit && !format.Plain { + mode, err = report.SelectMode(format, a.terminalState()) + if err != nil { + return err + } + } if opts.Rule != "" && !a.Engine.HasRule(opts.Rule) { return fmt.Errorf("rule %s is not registered in this build", opts.Rule) } - document, err := a.diagnose(cmd.Context(), opts) if err != nil { return toolFailure{err} } - - if opts.JSON { - if err := report.WriteJSON(a.Stdout, document); err != nil { - return toolFailure{err} - } - } else if err := report.WriteText(a.Stdout, document); err != nil { + switch mode { + case report.JSON: + err = report.WriteJSON(a.Stdout, document) + case report.Markdown: + err = report.WriteMarkdown(a.Stdout, document) + case report.Interactive: + err = a.Interactive(cmd.Context(), a.Stdin, a.Stdout, document) + default: + err = report.WriteText(a.Stdout, document) + } + if err != nil { return toolFailure{err} } - if opts.Verbose { - writeTimings(a, document) + if err := writeTimings(a, document); err != nil { + return toolFailure{err} + } } - a.exitCode = document.ExitCode return nil }, } - cmd.Flags().BoolVar(&opts.JSON, "json", false, "write one JSON document to stdout") + cmd.Flags().StringVar(&format.Format, "format", "", "output format: text, markdown or json") + cmd.Flags().BoolVar(&format.Plain, "plain", false, "force plain text without the terminal viewer") cmd.Flags().BoolVar(&opts.Verbose, "verbose", false, "write collector timings to stderr") cmd.Flags().StringVar(&opts.Component, "component", "", "only evaluate rules for this component") cmd.Flags().StringVar(&opts.Rule, "rule", "", "only evaluate this rule id (ENV001)") + cmd.Flags().StringVar(&opts.FromBundle, "from-bundle", "", "evaluate a sharing bundle offline; run no collectors") + bindCaptureFlags(cmd, &opts) + return cmd +} + +func bindCaptureFlags(cmd *cobra.Command, opts *config.Options) { cmd.Flags().IntSliceVar(&opts.Ports, "port", nil, "TCP ports your application needs (repeatable or comma separated)") cmd.Flags().StringVar(&opts.FixtureDir, "fixture-dir", "", "read snapshot.json in this directory; do not inspect this machine") cmd.Flags().BoolVar(&opts.ProbeNetwork, "probe-network", false, "probe configured proxy endpoints with a one-second total deadline") cmd.Flags().StringVar(&opts.PackageManager, "package-manager", "", "explicit project runner: npm, pnpm, yarn or bun, optionally @major.minor.patch") - - return cmd } func (a *App) diagnose(ctx context.Context, opts config.Options) (report.Document, error) { + if opts.FromBundle != "" { + capture, err := bundle.Load(ctx, opts.FromBundle) + if err != nil { + return report.Document{}, err + } + findings, err := a.Engine.Evaluate(capture.Snapshot, diagnostic.Filter{Component: opts.Component, RuleID: opts.Rule}) + if err != nil { + return report.Document{}, fmt.Errorf("evaluate rules: %w", err) + } + doc := report.NewDocument(a.Version, a.Now(), capture.Snapshot, findings, nil, len(a.Engine.Rules())) + doc.Source = &capture.Source + return doc, nil + } + return a.captureDocument(ctx, opts, true) +} + +func (a *App) captureDocument(ctx context.Context, opts config.Options, redact bool) (report.Document, error) { var snapshot collector.Snapshot var timings []collector.Timing if opts.FixtureDir != "" { @@ -95,32 +144,36 @@ func (a *App) diagnose(ctx context.Context, opts config.Options) (report.Documen snapshot, timings = runner.Run(ctx) snapshot.RequestedPorts = opts.Ports } - - home, _ := a.HomeDir() - snapshot.Redact(home) - for i := range timings { - if timings[i].Err != nil { - timings[i].Err = errors.New(collector.RedactText(timings[i].Err.Error(), home)) + if err := ctx.Err(); err != nil { + return report.Document{}, err + } + // Sharing export must see exact facts before home redaction can collapse + // distinct paths. Ordinary reports retain their existing M3 redaction. + if redact { + home, _ := a.HomeDir() + snapshot.Redact(home) + for i := range timings { + if timings[i].Err != nil { + timings[i].Err = errors.New(collector.RedactText(timings[i].Err.Error(), home)) + } } } - - findings, err := a.Engine.Evaluate(snapshot, diagnostic.Filter{ - Component: opts.Component, - RuleID: opts.Rule, - }) + findings, err := a.Engine.Evaluate(snapshot, diagnostic.Filter{Component: opts.Component, RuleID: opts.Rule}) if err != nil { return report.Document{}, fmt.Errorf("evaluate rules: %w", err) } - return report.NewDocument(a.Version, a.Now(), snapshot, findings, timings, len(a.Engine.Rules())), nil } -func writeTimings(a *App, document report.Document) { +func writeTimings(a *App, document report.Document) error { for _, c := range document.Collectors { status := "ok" if c.Error != "" { status = c.Error } - _, _ = fmt.Fprintf(a.Stderr, "collector %-10s %5dms %s\n", c.Name, c.DurationMs, status) + if _, err := fmt.Fprintf(a.Stderr, "collector %-10s %5dms %s\n", report.VisibleText(c.Name), c.DurationMs, report.VisibleText(status)); err != nil { + return fmt.Errorf("write collector timings: %w", err) + } } + return nil } diff --git a/internal/cli/m4_test.go b/internal/cli/m4_test.go new file mode 100644 index 0000000..33ae484 --- /dev/null +++ b/internal/cli/m4_test.go @@ -0,0 +1,330 @@ +package cli_test + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "io" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/shellghostt/envdoctor/internal/bundle" + "github.com/shellghostt/envdoctor/internal/cli" + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" +) + +type panicCollector struct{} + +func (panicCollector) Name() string { return "path" } +func (panicCollector) Collect(context.Context) (collector.Part, error) { + panic("offline diagnosis ran a collector") +} + +func TestM4CLIFormatAndTerminalMatrix(t *testing.T) { + capable := report.TerminalState{Input: true, Output: true, Error: true, Term: "xterm-256color"} + for _, tc := range []struct { + name string + flags []string + term report.TerminalState + interactive bool + kind string + }{ + {"default-viewer", nil, capable, true, "text"}, + {"no-tty", nil, report.TerminalState{}, false, "text"}, + {"stdout-redirect", nil, report.TerminalState{Input: true, Error: true}, false, "text"}, + {"stderr-redirect", nil, report.TerminalState{Input: true, Output: true}, false, "text"}, + {"stdin-redirect", nil, report.TerminalState{Output: true, Error: true}, false, "text"}, + {"dumb", nil, report.TerminalState{Input: true, Output: true, Error: true, Term: "dumb"}, false, "text"}, + {"empty-no-color", nil, report.TerminalState{Input: true, Output: true, Error: true, NoColor: true}, false, "text"}, + {"windows-unset-term", nil, report.TerminalState{Input: true, Output: true, Error: true}, true, "text"}, + {"plain", []string{"--plain"}, capable, false, "text"}, + {"text", []string{"--format", "text"}, capable, false, "text"}, + {"json", []string{"--format", "json"}, capable, false, "json"}, + {"json-alias", []string{"--json"}, capable, false, "json"}, + {"compatible-json", []string{"--json", "--format", "json"}, capable, false, "json"}, + {"markdown", []string{"--format", "markdown"}, capable, false, "markdown"}, + } { + t.Run(tc.name, func(t *testing.T) { + var out, errOut bytes.Buffer + app := newApp(&out, &errOut) + app.Engine = diagnostic.NewEngine(warnRule{severity: diagnostic.SeverityWarning}) + app.Terminals = func() report.TerminalState { return tc.term } + called := false + app.Interactive = func(_ context.Context, _ io.Reader, w io.Writer, d report.Document) error { + called = true + return report.WriteText(w, d) + } + code := cli.Main(append([]string{"envdoctor", "diagnose", "--verbose"}, tc.flags...), app) + if code != 1 || called != tc.interactive { + t.Fatalf("exit=%d viewer=%v stderr=%s", code, called, errOut.String()) + } + if strings.Contains(out.String(), "\x1b") || strings.Contains(out.String(), "collector platform") { + t.Fatal("stdout contaminated") + } + if !strings.Contains(errOut.String(), "collector platform") { + t.Fatal("verbose diagnostics missing") + } + switch tc.kind { + case "json": + validateAgainstSchema(t, compileDiagnoseSchema(t), out.Bytes()) + case "markdown": + if !strings.HasPrefix(out.String(), "# EnvDoctor diagnosis\n") || !strings.Contains(out.String(), "ENV003") { + t.Fatal("Markdown report missing") + } + default: + if !strings.Contains(out.String(), "Broken PATH entry") { + t.Fatal("plain report missing") + } + } + }) + } +} + +func TestM4CLIInvalidFormatsBeforeCollection(t *testing.T) { + for _, flags := range [][]string{{"--format", "html"}, {"--format", ""}, {"--json", "--format", "text"}, {"--json", "--format", "markdown"}, {"--plain", "--json"}, {"--plain", "--format", "markdown"}} { + var out, errOut bytes.Buffer + app := newApp(&out, &errOut) + app.Collectors = []collector.Collector{panicCollector{}} + app.Terminals = func() report.TerminalState { panic("format conflict inspected a terminal") } + if code := cli.Main(append([]string{"envdoctor", "diagnose"}, flags...), app); code != 4 || out.Len() != 0 { + t.Fatalf("exit=%d stdout=%s", code, out.String()) + } + } +} + +func TestM4CLIInteractiveFailurePreservesFallbackAndExit(t *testing.T) { + var out, errOut bytes.Buffer + app := newApp(&out, &errOut) + app.Terminals = func() report.TerminalState { return report.TerminalState{Input: true, Output: true, Error: true} } + app.Interactive = func(_ context.Context, _ io.Reader, w io.Writer, d report.Document) error { + if err := report.WriteText(w, d); err != nil { + return err + } + return context.Canceled + } + if code := cli.Main([]string{"envdoctor", "diagnose"}, app); code != 3 || !strings.Contains(out.String(), "environment diagnosis") { + t.Fatalf("exit=%d output=%s", code, out.String()) + } +} + +func offlineSnapshot() collector.Snapshot { + name, email := "SyntheticOfflineUser", "synthetic@private.invalid" + return collector.Snapshot{SchemaVersion: 1, CollectedAt: fixedNow, + Platform: &collector.Platform{OS: "windows", Version: "11", Arch: "amd64"}, Shell: &collector.Shell{Name: "powershell"}, + Path: &collector.PathInfo{Entries: []collector.PathEntry{{Raw: `C:\Users\SyntheticOfflineUser\gone`, Position: 0}}}, + Git: &collector.Git{Version: "2.53.0", UserName: &name, UserEmail: &email, IdentityChecked: true}, + Dependencies: &collector.Dependencies{ProjectRoot: `C:\Users\SyntheticOfflineUser\project`}, + } +} + +func bundleFile(t *testing.T, snapshot collector.Snapshot) string { + t.Helper() + findings, err := diagnostic.DefaultEngine().Evaluate(snapshot, diagnostic.Filter{}) + if err != nil { + t.Fatal(err) + } + doc := report.NewDocument("0.1.0", fixedNow, snapshot, findings, nil, 12) + data, err := bundle.Build(context.Background(), doc, bundle.Options{IncludeLogs: true}) + if err != nil { + t.Fatal(err) + } + path := filepath.Join(t.TempDir(), "capture.zip") + if err := os.WriteFile(path, data, 0600); err != nil { + t.Fatal(err) + } + return path +} + +func offlineApp(out, errOut *bytes.Buffer) cli.App { + app := newApp(out, errOut) + app.Version = "0.2.0" + app.Engine = diagnostic.DefaultEngine() + app.Collectors = []collector.Collector{panicCollector{}} + app.HomeDir = func() (string, error) { panic("offline diagnosis inspected the home directory") } + app.Getenv = func(string) string { panic("explicit offline format read the environment") } + app.LookupEnv = func(string) (string, bool) { panic("explicit offline format read the environment") } + return app +} + +func TestM4CLIOfflineEvidenceAndProvenance(t *testing.T) { + path := bundleFile(t, offlineSnapshot()) + for _, flags := range [][]string{{"--json"}, {"--format", "markdown"}, {"--plain"}, {"--format", "text"}, {"--json", "--rule", "ENV003", "--verbose"}, {"--json", "--component", "docker"}} { + var out, errOut bytes.Buffer + code := cli.Main(append([]string{"envdoctor", "diagnose", "--from-bundle", path}, flags...), offlineApp(&out, &errOut)) + want := 1 + if strings.Contains(strings.Join(flags, " "), "--component docker") { + want = 0 + } + if code != want { + t.Fatalf("exit=%d stderr=%s", code, errOut.String()) + } + if strings.Contains(out.String(), "SyntheticOfflineUser") || strings.Contains(out.String(), "synthetic@private.invalid") { + t.Fatal("original identity leaked") + } + if flags[0] == "--json" { + var doc report.Document + if err := json.Unmarshal(out.Bytes(), &doc); err != nil { + t.Fatal(err) + } + if doc.Source == nil || doc.Source.CaptureTool.Version != "0.1.0" || doc.Tool.Version != "0.2.0" || !doc.Snapshot.CollectedAt.Equal(fixedNow) { + t.Fatal("capture/evaluator provenance missing") + } + validateAgainstSchema(t, compileDiagnoseSchema(t), out.Bytes()) + } + } +} + +func TestM4CLIOfflineAllExitCodes(t *testing.T) { + for code := 0; code <= 3; code++ { + s := offlineSnapshot() + s.Path.Entries[0].Exists = true + s.Path.Entries[0].IsDir = true + switch code { + case 1: + s.Path.Entries[0].Exists = false + s.Path.Entries[0].IsDir = false + case 2: + s.Docker = &collector.Docker{Installed: true, CLIVersion: "29.9.2", ProbeStatus: "unavailable"} + case 3: + s.Errors = []collector.CollectorError{{Collector: "runtimes", Message: "secret failure"}} + } + var out, errOut bytes.Buffer + if got := cli.Main([]string{"envdoctor", "diagnose", "--from-bundle", bundleFile(t, s), "--json"}, offlineApp(&out, &errOut)); got != code { + t.Fatalf("got=%d want=%d stderr=%s", got, code, errOut.String()) + } + } +} + +func TestM4CLIConflictsAndReadFailuresAreOffline(t *testing.T) { + missing := filepath.Join(t.TempDir(), "SECRET-PATH-not-present.zip") + for _, extra := range [][]string{{"--fixture-dir", "SECRET-PATH"}, {"--port", "3000"}, {"--probe-network"}, {"--probe-network=false"}, {"--package-manager", "npm"}, {"--fixture-dir="}} { + var out, errOut bytes.Buffer + code := cli.Main(append([]string{"envdoctor", "diagnose", "--from-bundle", missing, "--json"}, extra...), offlineApp(&out, &errOut)) + if code != 4 || out.Len() != 0 || strings.Contains(errOut.String(), "SECRET-PATH") { + t.Fatalf("conflict exit=%d stderr=%s", code, errOut.String()) + } + } + for _, path := range []string{missing, filepath.Join(t.TempDir(), "SECRET-PATH-invalid.zip")} { + if path != missing { + if err := os.WriteFile(path, []byte("SECRET-CAPTURED-ERROR"), 0600); err != nil { + t.Fatal(err) + } + } + var out, errOut bytes.Buffer + if code := cli.Main([]string{"envdoctor", "diagnose", "--from-bundle", path, "--json"}, offlineApp(&out, &errOut)); code != 3 || out.Len() != 0 || strings.Contains(errOut.String(), "SECRET-") { + t.Fatalf("invalid input exit=%d stderr=%s", code, errOut.String()) + } + } +} + +func TestM4CLIBundleWriterSuccessAndFailures(t *testing.T) { + dir := t.TempDir() + fixture := filepath.Join(dir, "fixture") + if err := os.Mkdir(fixture, 0700); err != nil { + t.Fatal(err) + } + raw, err := json.Marshal(offlineSnapshot()) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(fixture, "snapshot.json"), raw, 0600); err != nil { + t.Fatal(err) + } + destination := filepath.Join(dir, "SECRET-OUTPUT.zip") + var out, errOut bytes.Buffer + app := offlineApp(&out, &errOut) + code := cli.Main([]string{"envdoctor", "bundle", "--fixture-dir", fixture, "-o", destination, "--include-logs"}, app) + if code != 0 || errOut.Len() != 0 || !strings.Contains(out.String(), "original diagnosis exit 1") || strings.Contains(out.String(), "SECRET-") { + t.Fatalf("exit=%d stdout=%s stderr=%s", code, out.String(), errOut.String()) + } + if _, err := bundle.Load(context.Background(), destination); err != nil { + t.Fatal(err) + } + for _, flags := range [][]string{{"--output", ""}, {"--json"}, {"--rule", "ENV003"}, {"--from-bundle", destination}, {"--fixture-dir", fixture, "--probe-network"}} { + out.Reset() + errOut.Reset() + if code := cli.Main(append([]string{"envdoctor", "bundle"}, flags...), offlineApp(&out, &errOut)); code != 4 { + t.Fatalf("usage exit=%d stderr=%s", code, errOut.String()) + } + } + out.Reset() + errOut.Reset() + if code := cli.Main([]string{"envdoctor", "bundle", "--fixture-dir", fixture, "--output", dir}, offlineApp(&out, &errOut)); code != 3 { + t.Fatalf("I/O exit=%d stderr=%s", code, errOut.String()) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + out.Reset() + errOut.Reset() + app = offlineApp(&out, &errOut) + app.Context = ctx + if code := cli.Main([]string{"envdoctor", "bundle", "--fixture-dir", fixture, "--output", destination}, app); code != 3 { + t.Fatalf("cancel exit=%d", code) + } +} + +type m4WriteFailure struct{} + +func (m4WriteFailure) Write([]byte) (int, error) { return 0, errors.New("SECRET-WRITER-ERROR") } + +type m4WriteShort struct{} + +func (m4WriteShort) Write(p []byte) (int, error) { return max(0, len(p)-1), nil } + +func TestM4CLIBundleStatusFailurePreservesDestination(t *testing.T) { + for _, writer := range []struct { + name string + output io.Writer + }{{"failure", m4WriteFailure{}}, {"short-write", m4WriteShort{}}} { + t.Run(writer.name, func(t *testing.T) { + dir := t.TempDir() + fixture := filepath.Join(dir, "fixture") + if err := os.Mkdir(fixture, 0700); err != nil { + t.Fatal(err) + } + raw, err := json.Marshal(offlineSnapshot()) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(fixture, "snapshot.json"), raw, 0600); err != nil { + t.Fatal(err) + } + destination := filepath.Join(dir, "previous.zip") + previous := []byte("previous capture must survive a failed status write") + if err := os.WriteFile(destination, previous, 0600); err != nil { + t.Fatal(err) + } + var out, errOut bytes.Buffer + app := offlineApp(&out, &errOut) + app.Stdout = writer.output + if code := cli.Main([]string{"envdoctor", "bundle", "--fixture-dir", fixture, "--output", destination}, app); code != 3 || strings.Contains(errOut.String(), "SECRET-") { + t.Fatalf("exit=%d stderr=%s", code, errOut.String()) + } + got, err := os.ReadFile(destination) + if err != nil || !bytes.Equal(got, previous) { + t.Fatalf("previous destination changed: %q, %v", got, err) + } + entries, err := os.ReadDir(dir) + if err != nil || len(entries) != 2 { + t.Fatalf("owned staging file leaked: %v, %v", entries, err) + } + }) + } +} + +func TestM4CLIStructuredOutputErrors(t *testing.T) { + path := bundleFile(t, offlineSnapshot()) + for _, format := range []string{"text", "markdown", "json"} { + var out, errOut bytes.Buffer + app := offlineApp(&out, &errOut) + app.Stdout = m4WriteFailure{} + if code := cli.Main([]string{"envdoctor", "diagnose", "--from-bundle", path, "--format", format}, app); code != 3 || strings.Contains(errOut.String(), "SECRET-") { + t.Fatalf("exit=%d stderr=%s", code, errOut.String()) + } + } +} diff --git a/internal/config/config.go b/internal/config/config.go index 3a10fb4..edac371 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -35,6 +35,7 @@ type Options struct { Rule string Ports []int FixtureDir string + FromBundle string ProbeNetwork bool PackageManager string } @@ -43,6 +44,9 @@ type Options struct { // check whether the rule exists: that needs the rule registry, which the // caller owns. func (o Options) Validate() error { + if o.FromBundle != "" && (o.FixtureDir != "" || len(o.Ports) > 0 || o.ProbeNetwork || o.PackageManager != "") { + return fmt.Errorf("--from-bundle conflicts with live capture flags and --fixture-dir") + } if o.ProbeNetwork && o.FixtureDir != "" { return fmt.Errorf("--probe-network cannot be used with --fixture-dir") } diff --git a/internal/report/document.go b/internal/report/document.go index 37c0984..2d95735 100644 --- a/internal/report/document.go +++ b/internal/report/document.go @@ -21,6 +21,22 @@ type Document struct { ExitCode int `json:"exitCode"` Rules int `json:"rules"` Collectors []Collector `json:"collectors"` + Source *Source `json:"source,omitempty"` +} + +// Source is present only for bundle evidence, never for ordinary M3 output. +type Source struct { + Kind string `json:"kind"` + CaptureTool Tool `json:"captureTool"` + CapturedAt time.Time `json:"capturedAt"` + RedactionPolicy string `json:"redactionPolicy"` + CaptureExitCode int `json:"captureExitCode"` + Omissions []Omission `json:"omissions"` +} + +type Omission struct { + Field string `json:"field"` + Reason string `json:"reason"` } // Tool identifies the binary that produced the document. diff --git a/internal/report/format.go b/internal/report/format.go new file mode 100644 index 0000000..642cd87 --- /dev/null +++ b/internal/report/format.go @@ -0,0 +1,70 @@ +package report + +import ( + "fmt" + "strings" + "unicode" +) + +type Mode string + +const ( + Text Mode = "text" + JSON Mode = "json" + Markdown Mode = "markdown" + Interactive Mode = "interactive" +) + +type FormatOptions struct { + Format string + Explicit bool + JSON bool + Plain bool +} + +// TerminalState is supplied by the CLI. Selection itself never inspects a host. +type TerminalState struct { + Input, Output, Error bool + Term string + NoColor bool +} + +func SelectMode(o FormatOptions, terminal TerminalState) (Mode, error) { + if o.Explicit && o.Format != string(Text) && o.Format != string(JSON) && o.Format != string(Markdown) { + return "", fmt.Errorf("--format must be text, markdown or json") + } + if o.JSON && o.Explicit && o.Format != string(JSON) { + return "", fmt.Errorf("--json conflicts with --format") + } + if o.Plain && (o.JSON || o.Explicit && o.Format != string(Text)) { + return "", fmt.Errorf("--plain requires text output") + } + if o.JSON { + return JSON, nil + } + if o.Explicit { + return Mode(o.Format), nil + } + if o.Plain || terminal.NoColor || strings.EqualFold(strings.TrimSpace(terminal.Term), "dumb") || !terminal.Input || !terminal.Output || !terminal.Error { + return Text, nil + } + return Interactive, nil +} + +// VisibleText preserves ordinary report bytes and makes terminal controls data. +func VisibleText(s string) string { + if !strings.ContainsFunc(s, func(r rune) bool { return r != '\n' && r != '\t' && displayControl(r) }) { + return s + } + var b strings.Builder + for _, r := range s { + if r != '\n' && r != '\t' && displayControl(r) { + fmt.Fprintf(&b, "\\u%04X", r) + } else { + b.WriteRune(r) + } + } + return b.String() +} + +func displayControl(r rune) bool { return unicode.IsControl(r) || unicode.Is(unicode.Cf, r) } diff --git a/internal/report/interactive.go b/internal/report/interactive.go new file mode 100644 index 0000000..94cf916 --- /dev/null +++ b/internal/report/interactive.go @@ -0,0 +1,230 @@ +package report + +import ( + "bytes" + "context" + "errors" + "fmt" + "io" + "strings" + "sync" + "time" + + tea "github.com/charmbracelet/bubbletea" + "github.com/charmbracelet/x/ansi" + "golang.org/x/term" +) + +type viewer struct { + text string + lines []string + width int + height int + top int + interrupted bool +} + +func newViewer(text string) *viewer { + v := &viewer{text: strings.ReplaceAll(text, "\t", " "), width: 80, height: 24} + v.resize(80, 24) + return v +} + +func (v *viewer) resize(width, height int) { + v.width, v.height = max(1, min(width, 512)), max(1, min(height, 256)) + v.lines = strings.Split(ansi.Hardwrap(v.text, v.width, false), "\n") + v.top = min(v.top, max(0, len(v.lines)-max(1, v.height-2))) +} + +func (*viewer) Init() tea.Cmd { return nil } + +func (v *viewer) Update(msg tea.Msg) (tea.Model, tea.Cmd) { + switch msg := msg.(type) { + case tea.WindowSizeMsg: + if max(1, min(msg.Width, 512)) != v.width || max(1, min(msg.Height, 256)) != v.height { + v.resize(msg.Width, msg.Height) + } + case tea.KeyMsg: + switch msg.String() { + case "q", "esc": + return v, tea.Quit + case "ctrl+c": + v.interrupted = true + return v, tea.Quit + case "up", "k": + v.top-- + case "down", "j": + v.top++ + case "pgup": + v.top -= max(1, v.height-2) + case "pgdown", " ": + v.top += max(1, v.height-2) + case "home", "g": + v.top = 0 + case "end", "G": + v.top = len(v.lines) + } + v.top = max(0, min(v.top, max(0, len(v.lines)-max(1, v.height-2)))) + } + return v, nil +} + +func (v *viewer) View() string { + header := ansi.Truncate("EnvDoctor | captured facts and manual advice", v.width, "") + if v.height == 1 { + return header + } + footer := ansi.Truncate(fmt.Sprintf("↑/↓ scroll · pgup/down · q exit | %d/%d", v.top+1, len(v.lines)), v.width, "") + if v.height == 2 { + return header + "\n" + footer + } + end := min(len(v.lines), v.top+v.height-2) + lines := make([]string, 0, end-v.top) + for _, line := range v.lines[v.top:end] { + line = ansi.Truncate(line, v.width, "") + for _, style := range []struct{ prefix, color string }{{"[CRITICAL]", "35"}, {"[ERROR]", "31"}, {"[WARNING]", "33"}, {"[INFO]", "34"}} { + if strings.HasPrefix(line, style.prefix) { + line = "\x1b[1;" + style.color + "m" + line + "\x1b[0m" + break + } + } + lines = append(lines, line) + } + return "\x1b[1;36m" + header + "\x1b[0m\n" + strings.Join(lines, "\n") + "\n\x1b[2m" + footer + "\x1b[0m" +} + +// Bubble Tea's renderer can ignore Write errors. Record them and cancel its +// context so an output failure cannot leave an interactive event loop running. +type terminalWriter struct { + output io.Writer + cancel context.CancelFunc + mu sync.Mutex + err error +} + +func (w *terminalWriter) Fd() uintptr { + if file, ok := w.output.(interface{ Fd() uintptr }); ok { + return file.Fd() + } + return ^uintptr(0) +} + +func (w *terminalWriter) Write(data []byte) (int, error) { + w.mu.Lock() + defer w.mu.Unlock() + n, err := w.output.Write(data) + if err == nil && n != len(data) { + err = io.ErrShortWrite + } + if err != nil { + if w.err == nil { + w.err = err + } + w.cancel() + } + return n, err +} + +type terminalState struct { + fd int + state *term.State +} + +func saveTerminalStates(streams ...any) ([]terminalState, error) { + states := []terminalState{} + seen := map[int]bool{} + for _, stream := range streams { + file, ok := stream.(interface{ Fd() uintptr }) + if !ok { + continue + } + fd := int(file.Fd()) + if seen[fd] || !term.IsTerminal(fd) { + continue + } + state, err := term.GetState(fd) + if err != nil { + return nil, err + } + states = append(states, terminalState{fd: fd, state: state}) + seen[fd] = true + } + return states, nil +} + +func restoreTerminalStates(states []terminalState) error { + var result error + for _, saved := range states { + result = errors.Join(result, term.Restore(saved.fd, saved.state)) + } + return result +} + +func resizePoller(ctx context.Context, request func(), interval time.Duration) <-chan struct{} { + done := make(chan struct{}) + go func() { + defer close(done) + ticker := time.NewTicker(interval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + request() + } + } + }() + return done +} + +func (w *terminalWriter) failure() error { + w.mu.Lock() + defer w.mu.Unlock() + return w.err +} + +// WriteInteractive always leaves a plain report after the terminal viewer. +// The caller selects this path only for capable terminal streams. +func WriteInteractive(ctx context.Context, input io.Reader, output io.Writer, doc Document) error { + var plain bytes.Buffer + if err := WriteText(&plain, doc); err != nil { + return err + } + states, saveErr := saveTerminalStates(input, output) + if saveErr != nil { + return errors.Join(saveErr, WriteText(output, doc)) + } + runCtx, cancel := context.WithCancel(ctx) + defer cancel() + w := &terminalWriter{output: output, cancel: cancel} + v := newViewer(plain.String()) + program := tea.NewProgram(v, tea.WithInput(input), tea.WithOutput(w), tea.WithContext(runCtx), + tea.WithAltScreen(), tea.WithoutSignalHandler(), tea.WithoutBracketedPaste()) + // Bubble Tea v1.3.10 only checks the initial size on Windows (no SIGWINCH). + // Poll through its window-size command on terminal outputs. The adapter owns + // and joins this context-cancelled timer on every return, including startup + // and output failure; no Tick command remains asleep after the viewer exits. + var resizeDone <-chan struct{} + if file, ok := output.(interface{ Fd() uintptr }); ok && term.IsTerminal(int(file.Fd())) { + resizeDone = resizePoller(runCtx, func() { program.Send(tea.WindowSize()()) }, 200*time.Millisecond) + } + _, runErr := program.Run() + cancel() + if resizeDone != nil { + <-resizeDone + } + var cleanupErr error + if runErr != nil && input != nil { + cleanupErr = program.ReleaseTerminal() + } + cleanupErr = errors.Join(cleanupErr, restoreTerminalStates(states)) + if v.interrupted { + runErr = errors.Join(runErr, context.Canceled) + } + writeErr := WriteText(output, doc) + if err := errors.Join(runErr, cleanupErr, w.failure(), writeErr); err != nil { + return fmt.Errorf("interactive report failed; plain fallback attempted: %w", err) + } + return nil +} diff --git a/internal/report/interactive_test.go b/internal/report/interactive_test.go new file mode 100644 index 0000000..6247bb7 --- /dev/null +++ b/internal/report/interactive_test.go @@ -0,0 +1,167 @@ +package report + +import ( + "bytes" + "context" + "errors" + "runtime" + "strings" + "testing" + "time" + + tea "github.com/charmbracelet/bubbletea" + "github.com/charmbracelet/x/ansi" + + "github.com/shellghostt/envdoctor/internal/collector" +) + +func TestM4ViewerNarrowResizeScrollAndInterrupt(t *testing.T) { + v := newViewer(strings.Repeat("wide 界 text \t evidence\n", 50)) + for _, size := range [][2]int{{1, 1}, {2, 2}, {12, 8}, {80, 24}, {0, 0}, {10000, 10000}} { + _, _ = v.Update(tea.WindowSizeMsg{Width: size[0], Height: size[1]}) + for _, key := range []tea.KeyMsg{{Type: tea.KeyEnd}, {Type: tea.KeyPgUp}, {Type: tea.KeyDown}, {Type: tea.KeyHome}, {Type: tea.KeyUp}, {Type: tea.KeyPgDown}} { + _, _ = v.Update(key) + } + lines := strings.Split(v.View(), "\n") + if len(lines) > v.height || v.top < 0 || v.top > max(0, len(v.lines)-max(1, v.height-2)) { + t.Fatalf("viewport out of bounds at %v", size) + } + for _, line := range lines { + if width := ansi.StringWidth(line); width > v.width { + t.Fatalf("line width=%d viewport=%d: %q", width, v.width, line) + } + } + } + _, cmd := v.Update(tea.KeyMsg{Type: tea.KeyCtrlC}) + if !v.interrupted || cmd == nil { + t.Fatal("interrupt did not quit") + } + if _, ok := cmd().(tea.QuitMsg); !ok { + t.Fatal("interrupt did not return quit command") + } +} + +type lifecycleWriter struct { + bytes bytes.Buffer + cancel context.CancelFunc + failFrame bool + failed bool +} + +func (w *lifecycleWriter) Write(p []byte) (int, error) { + if w.cancel != nil && bytes.Contains(p, []byte("\x1b[?1049h")) { + w.cancel() + } + if w.failFrame && !w.failed && bytes.Contains(p, []byte("EnvDoctor |")) { + w.failed = true + return 0, errors.New("synthetic frame failure") + } + return w.bytes.Write(p) +} + +func viewerDocument() Document { + stamp := time.Date(2026, 11, 19, 10, 0, 0, 0, time.UTC) + return NewDocument("0.1.0", stamp, collector.Snapshot{SchemaVersion: 1, CollectedAt: stamp, Platform: &collector.Platform{OS: "windows", Arch: "amd64", Version: "11"}}, nil, nil, 12) +} + +func TestM4InteractiveQuitCancellationAndOutputCleanup(t *testing.T) { + for _, name := range []string{"quit", "cancel", "output-failure"} { + t.Run(name, func(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + writer := &lifecycleWriter{} + var input *strings.Reader + if name == "quit" { + input = strings.NewReader("q") + } + if name == "cancel" { + writer.cancel = cancel + } + if name == "output-failure" { + writer.failFrame = true + } + var err error + if input != nil { + err = WriteInteractive(ctx, input, writer, viewerDocument()) + } else { + err = WriteInteractive(ctx, nil, writer, viewerDocument()) + } + if (err != nil) != (name != "quit") { + t.Fatalf("%s error=%v", name, err) + } + if name == "cancel" && !errors.Is(err, context.Canceled) { + t.Fatalf("cancel=%v", err) + } + if name == "output-failure" && !writer.failed { + t.Fatal("frame failure was never exercised") + } + output := writer.bytes.String() + leave := strings.LastIndex(output, "\x1b[?1049l") + plain := strings.LastIndex(output, "EnvDoctor 0.1.0 - environment diagnosis") + if !strings.Contains(output, "\x1b[?1049h") || leave < 0 || !strings.Contains(output, "\x1b[?25h") || plain < leave { + t.Fatalf("screen/cursor/fallback cleanup missing: %q", output) + } + }) + } +} + +func TestM4RepeatedViewerLifecycleReleasesResources(t *testing.T) { + doc := viewerDocument() + if err := WriteInteractive(context.Background(), strings.NewReader("q"), &lifecycleWriter{}, doc); err != nil { + t.Fatal(err) + } + runtime.GC() + var before, after runtime.MemStats + runtime.ReadMemStats(&before) + beforeG := runtime.NumGoroutine() + start := time.Now() + for i := 0; i < 50; i++ { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + writer := &lifecycleWriter{} + if i%2 == 0 { + if err := WriteInteractive(ctx, strings.NewReader("q"), writer, doc); err != nil { + t.Fatal(err) + } + } else { + writer.cancel = cancel + if err := WriteInteractive(ctx, nil, writer, doc); !errors.Is(err, context.Canceled) { + t.Fatalf("cancel=%v", err) + } + } + cancel() + } + deadline := time.Now().Add(2 * time.Second) + for runtime.NumGoroutine() > beforeG+2 && time.Now().Before(deadline) { + runtime.GC() + time.Sleep(10 * time.Millisecond) + } + runtime.GC() + runtime.ReadMemStats(&after) + afterG := runtime.NumGoroutine() + t.Logf("50 viewers: %s; goroutines %d -> %d; retained heap %d -> %d bytes", time.Since(start), beforeG, afterG, before.HeapAlloc, after.HeapAlloc) + if afterG > beforeG+2 || int64(after.HeapAlloc)-int64(before.HeapAlloc) > 2<<20 { + t.Fatal("viewer resources retained after repeated exits") + } +} + +func TestM4ResizePollerStopsAndJoinsOnCancellation(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + requested := make(chan struct{}, 1) + done := resizePoller(ctx, func() { + select { + case requested <- struct{}{}: + default: + } + }, time.Millisecond) + select { + case <-requested: + case <-time.After(time.Second): + t.Fatal("resize was not requested") + } + cancel() + select { + case <-done: + case <-time.After(time.Second): + t.Fatal("resize timer/goroutine retained") + } +} diff --git a/internal/report/json.go b/internal/report/json.go index ecb12a4..ac32544 100644 --- a/internal/report/json.go +++ b/internal/report/json.go @@ -9,7 +9,7 @@ import ( // WriteJSON writes exactly one JSON document followed by a newline. Nothing // else is written to the stream; progress and warnings go to stderr. func WriteJSON(w io.Writer, document Document) error { - encoder := json.NewEncoder(w) + encoder := json.NewEncoder(checkedWriter{output: w}) encoder.SetEscapeHTML(false) encoder.SetIndent("", " ") if err := encoder.Encode(document); err != nil { @@ -17,3 +17,17 @@ func WriteJSON(w io.Writer, document Document) error { } return nil } + +type checkedWriter struct{ output io.Writer } + +func (w checkedWriter) Write(p []byte) (int, error) { + n, err := w.output.Write(p) + if err == nil && n != len(p) { + err = io.ErrShortWrite + } + return n, err +} +func writeString(w io.Writer, s string) error { + _, err := checkedWriter{output: w}.Write([]byte(s)) + return err +} diff --git a/internal/report/m4_test.go b/internal/report/m4_test.go new file mode 100644 index 0000000..54e843d --- /dev/null +++ b/internal/report/m4_test.go @@ -0,0 +1,144 @@ +package report_test + +import ( + "bytes" + "errors" + "io" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/shellghostt/envdoctor/internal/collector" + "github.com/shellghostt/envdoctor/internal/diagnostic" + "github.com/shellghostt/envdoctor/internal/report" +) + +type failingWriter struct{} + +func (failingWriter) Write([]byte) (int, error) { return 0, errors.New("synthetic write failure") } + +func TestM4FormatPolicy(t *testing.T) { + tty := report.TerminalState{Input: true, Output: true, Error: true, Term: "xterm-256color"} + for _, tc := range []struct { + name string + opts report.FormatOptions + term report.TerminalState + want report.Mode + bad bool + }{ + {"interactive", report.FormatOptions{}, tty, report.Interactive, false}, + {"no-terminal", report.FormatOptions{}, report.TerminalState{}, report.Text, false}, + {"redirect-stdout", report.FormatOptions{}, report.TerminalState{Input: true, Error: true}, report.Text, false}, + {"redirect-stderr", report.FormatOptions{}, report.TerminalState{Input: true, Output: true}, report.Text, false}, + {"redirect-stdin", report.FormatOptions{}, report.TerminalState{Output: true, Error: true}, report.Text, false}, + {"dumb", report.FormatOptions{}, report.TerminalState{Input: true, Output: true, Error: true, Term: "dumb"}, report.Text, false}, + {"no-color-even-empty", report.FormatOptions{}, report.TerminalState{Input: true, Output: true, Error: true, NoColor: true}, report.Text, false}, + {"windows-no-term", report.FormatOptions{}, report.TerminalState{Input: true, Output: true, Error: true}, report.Interactive, false}, + {"plain", report.FormatOptions{Plain: true}, tty, report.Text, false}, + {"explicit-text", report.FormatOptions{Format: "text", Explicit: true}, tty, report.Text, false}, + {"json-alias", report.FormatOptions{JSON: true}, tty, report.JSON, false}, + {"json", report.FormatOptions{Format: "json", Explicit: true}, tty, report.JSON, false}, + {"markdown", report.FormatOptions{Format: "markdown", Explicit: true}, tty, report.Markdown, false}, + {"compatible-json", report.FormatOptions{JSON: true, Format: "json", Explicit: true}, tty, report.JSON, false}, + {"conflicting-alias", report.FormatOptions{JSON: true, Format: "markdown", Explicit: true}, tty, "", true}, + {"plain-json", report.FormatOptions{JSON: true, Plain: true}, tty, "", true}, + {"plain-markdown", report.FormatOptions{Plain: true, Format: "markdown", Explicit: true}, tty, "", true}, + {"unknown", report.FormatOptions{Format: "html", Explicit: true}, tty, "", true}, + } { + t.Run(tc.name, func(t *testing.T) { + got, err := report.SelectMode(tc.opts, tc.term) + if (err != nil) != tc.bad || !tc.bad && got != tc.want { + t.Fatalf("mode=%q, err=%v; want=%q, bad=%v", got, err, tc.want, tc.bad) + } + }) + } +} + +func markdownDocument() report.Document { + s := collector.Snapshot{SchemaVersion: 1, CollectedAt: fixedTime, + Platform: &collector.Platform{OS: "windows", Arch: "amd64"}, + Errors: []collector.CollectorError{{Collector: "git", Message: "captured collector unavailable"}}} + findings := []diagnostic.Finding{ + {RuleID: "ENV900", Severity: diagnostic.SeverityCritical, Component: "git", Title: "Critical evidence", + Detected: []string{"|[x](https://invalid.example)"}, Cause: "Unknown cause", + Fix: &diagnostic.Fix{Summary: "Check manually", Commands: []string{"echo `test`"}}, Verify: "echo \"ok\""}, + {RuleID: "ENV901", Severity: diagnostic.SeverityError, Component: "git", Title: "Error evidence", Detected: []string{"error"}}, + {RuleID: "ENV902", Severity: diagnostic.SeverityWarning, Component: "git", Title: "Warning evidence", Detected: []string{"warning"}}, + {RuleID: "ENV903", Severity: diagnostic.SeverityInfo, Component: "git", Title: "Info evidence", Detected: []string{"info"}}, + } + findings[0].Detected = append(findings[0].Detected, `C:\synthetic\project with spaces\`+strings.Repeat(`long-directory-name-0123456789\`, 8)+`|file.txt`) + doc := report.NewDocument("0.1.0", fixedTime, s, findings, nil, 12) + doc.Source = &report.Source{Kind: "bundle", CaptureTool: report.Tool{Name: "envdoctor", Version: "0.1.0"}, + CapturedAt: fixedTime, RedactionPolicy: "allowlist-v1", CaptureExitCode: 2, + Omissions: []report.Omission{{Field: "git.identity", Reason: "identifying_values"}}} + return doc +} + +func TestM4MarkdownBoundsAndBidiControls(t *testing.T) { + doc := markdownDocument() + doc.Findings[0].Detected = []string{"visible\u202Eafter"} + var out bytes.Buffer + if err := report.WriteMarkdown(&out, doc); err != nil { + t.Fatal(err) + } + if strings.ContainsRune(out.String(), '\u202E') || !strings.Contains(out.String(), "u202E") { + t.Fatal("bidi control not made visible") + } + doc.Findings[0].Detected = []string{strings.Repeat("!", 1<<20)} + out.Reset() + if err := report.WriteMarkdown(&out, doc); err == nil || out.Len() != 0 { + t.Fatal("oversized Markdown partially written or accepted") + } +} + +func TestM4MarkdownGoldenAndEscaping(t *testing.T) { + var out bytes.Buffer + if err := report.WriteMarkdown(&out, markdownDocument()); err != nil { + t.Fatal(err) + } + want, err := os.ReadFile(filepath.Join("testdata", "markdown-m4.golden")) + if err != nil { + t.Fatal(err) + } + if out.String() != string(want) { + t.Fatalf("Markdown differs from reviewed golden:\n%s", out.String()) + } + for _, forbidden := range []string{"