scanner: Harden detection and scan coverage

[pixincreate]

Summary

• Match complete credentials and remove unsafe exemptions, credit-card detection, and luhn validation.
• Fix Git framing, historical blob scans, submodule handling, and filesystem visit limits.
• Separate scan coverage from findings and enforce bounded inputs and output safety.
• Fix CI fixtures, preserve portable filename decoding tests, and add paired false-positive regressions.
• Shorten the README and remove the review documents.

Compatibility

Reports can return INCOMPLETE; --fail-on-unscannable applies in every exit mode.
Corrected matches can change baseline fingerprints, but the baseline schema stays unchanged.
Incomplete scans cannot update baselines.
Input, line, path, and finding limits can stop oversized scans.

Validation

• Local debug and release suites each pass 320 tests.
• Clippy with -D warnings, formatting, and Rust 1.85 checks pass.
• Local self-scan passes with .DS_Store excluded; baseline regeneration produces no drift.
• The synthetic corpus contains 39 cases: 20 credentials and 19 non-credentials.
• The latest fixes leave the baseline unchanged.

Check the PR checks for cross-platform CI results.
The synthetic corpus does not establish production precision or recall.
Large workspace scans can reach the finding budget and exit 2 without a report.
Container execution and production runtime validation remain outside the tested scope.

[Copilot]

Copilot review overview

🟡 Changes recommended

Atomic writing, Git subprocess scaling, and staged line-limit handling have unresolved correctness and performance issues.

Review effort: Balanced
Findings: 2 · 2

Open (4)

• Update test fixture for world-writable temporary directory rejection · New
• Support atomic replacement of existing files on Windows · New
• Exclude Git framing and terminators from line length limits · New
• Batch git cat-file size lookups to avoid per-file process overhead · New

What changed in this PR

Hardens secret detection, scan coverage reporting, resource limits, configuration validation, and output safety across the CLI, hooks, and GitHub Action.

Changes:

• Improves credential matching, Git scanning, coverage reporting, and SARIF output.
• Adds bounded resource handling and safer report/baseline writes.
• Expands regression, accuracy, integration, and workload validation.

File	Description
tests/​scanner_tests.rs	Tests complete credential matching.
tests/​report_tests.rs	Tests coverage status and SARIF URIs.
tests/​hooks_tests.rs	Tests fail-closed pre-push behavior.
tests/​hardening_tests.rs	Adds end-to-end hardening regressions.
tests/​detector_tests.rs	Updates detector accuracy expectations.
tests/​baseline_tests.rs	Tests corrected baseline fingerprints.
tests/​accuracy_corpus.toml	Defines synthetic accuracy cases.
tests/​accuracy_corpus_tests.rs	Executes and measures the corpus.
templates/​pre-push.sh	Blocks incomplete push scans.
src/​utils.rs	Adds atomic protected file replacement.
src/​scanner/​staged.rs	Hardens Git diff and blob scanning.
src/​scanner/​lines.rs	Adds bounded complete-input scanning.
src/​scanner/​limits.rs	Defines resource budgets.
src/​scanner/​files.rs	Bounds traversal and reports skipped inputs.
src/​scanner/​error.rs	Adds resource-limit errors.
src/​scanner.rs	Integrates coverage, limits, and fingerprints.
src/​run_error.rs	Rejects incomplete baseline updates.
src/​report/​sarif.rs	Adds coverage fields and URI encoding.
src/​report.rs	Separates findings from coverage status.
src/​lib.rs	Enforces coverage exit policy.
src/​detector.rs	Removes Luhn and validates entropy.
src/​config/​tests/​application.rs	Updates validator configuration tests.
src/​config.rs	Rejects invalid detector configuration.
src/​cli.rs	Adds lockfile and coverage options.
src/​baseline.rs	Uses atomic baseline writes.
scripts/​benchmark.py	Measures bounded workloads.
scripts/​action_validation/​validate.py	Validates Action coverage enforcement.
scripts/​action_validation/​keywatch_action_scenarios.py	Adds incomplete-report scenarios.
README.md	Documents compatibility and security changes.
docs/​security-and-validation.md	Documents limits and validation evidence.
docs/​plans/​2026-10-06-hardening.md	Records the hardening plan.
detectors.toml	Refines built-in detection policy.
CHANGELOG.md	Records breaking and behavioral changes.
action.yml	Makes Action scans fail closed.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Valid submodule diffs can abort scanning, restrictive umasks break output permissions, and traversal can bypass the path budget.

5 open findings

• Path limit does not count total traversed entries · New
• Support atomic replacement of existing files on Windows
• Update test fixture for world-writable temporary directory rejection
• Rejects valid non-blob Git objects · New
• Umask can create replacement files with mode 000 · New

2 resolved since last review

• Batch git cat-file size lookups to avoid per-file process overhead
• Exclude Git framing and terminators from line length limits

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The shipped detector configuration is invalid TOML, and terminal control characters remain unsafe in human-readable output.

1 open finding

• Invalid TOML escape breaks detector loading and causes duplicate reports · New

5 resolved since last review

• Path limit does not count total traversed entries
• Support atomic replacement of existing files on Windows
• Update test fixture for world-writable temporary directory rejection
• Umask can create replacement files with mode 000
• Rejects valid non-blob Git objects

Previously missed (2)

In code that hasn't changed since last review

Unescaped control characters in file paths can manipulate terminal output

src/​scanner/​staged.rs:241

These newly decoded control bytes are retained in Finding.file_path, while non-verbose output interpolates that path directly into the terminal. A repository-controlled filename containing BEL, backspace, form feed, or vertical tab can therefore manipulate console output; keep the decoded path for Git lookup/report identity, but escape control characters at every human-readable output boundary.

Documentation overstates secret redaction for verbose show-secrets output

README.md:27

This is not true for --verbose --show-secrets: verbose mode prints the generated report to the console, and --show-secrets leaves matched content unredacted. Qualify this guarantee as applying to non-verbose console summaries so users do not expose secrets based on the documentation.

🧠 Review effort: Balanced