fix: stop disabling TruffleHog verification for verified-only scans

[lelia]

Summary

Stabilizes the TruffleHog experience for a single upcoming release, closing CE-351 and the last open item from CE-347.

1. Verified-only scans no longer disable verification (CE-351)

Turning Show Unverified Secrets off passed --no-verification, which disabled verification outright instead of returning only verified findings.

Severity is derived from each finding's Verified flag (trufflehog/__init__.py:476,481,541), so with verification disabled every finding came back Verified: false → low → non-blocking. On the setting's default path, no secret could ever block a run — the exact inverse of the intended behavior.

Verification now always runs; the setting selects result types only:

trufflehog_show_unverified	flag
off (default)	--results=verified
on	--results=verified,unverified,unknown

--include-detectors=all is now passed unconditionally so detector selection no longer changes as a side effect. TruffleHog already defaults to all detectors, so that half is a no-op in practice.

The setting is also now read through coerce_bool. Only the env loader coerces bool params (config.py:924); the dashboard normalizer assigns verbatim (config.py:1309) at higher priority, so a dashboard-supplied string "false" was truthy and would have reported unverified secrets to someone who turned them off. (Bugbot finding, fixed in 12048e0.)

2. A TruffleHog run that cannot scan no longer looks like a clean scan (CE-347)

Any non-zero exit was logged and converted into an empty result, so a malformed exclude pattern or a broken install silently zeroed out every secret finding while the run exited green. Non-zero exits and a missing binary now raise SystemExit naming the exit code and TruffleHog's own stderr.

This is the same failure class as #1 — secret scanning reporting nothing and passing — which is why it's folded in here rather than shipped separately. It also de-risks #1 directly: verification is now a live network call, so there are strictly more ways for the run to fail, and until now every one of them was silent.

SystemExit is deliberate — the connector manager catches Exception (manager.py:290), and this must not be downgraded to a skipped connector. It matches the fail-closed idiom already used for an unresolvable changed_files scope. The exclude-file cleanup in the finally block still runs on the failure path.

CE-347's other two criteria (multiple comma-separated patterns, **/appsettings.*.json globs) shipped in 2.2.1 via #94, which referenced CE-347 without closing it. This closes it.

Validation

• uv run --with pytest pytest -q — 349 passed
• Verified end-to-end against the pinned trufflesecurity/trufflehog:3.96.0 binary rather than from help text alone. Scanning a throwaway RSA private key:
  • --results=verified,unverified,unknown → PrivateKey Verified=False returned
  • --results=verified → zero findings
• Confirmed upstream's own default for --results is verified,unverified,unknown, so the "on" branch matches TruffleHog's default exactly.
• Smoke-tested socket-basics --help and importing both modules, to rule out a circular import from the new top-level from ...config import coerce_bool.

New tests/test_trufflehog_verification.py (10 tests): result-type flag per setting state, --no-verification absent in both, detector flag identical across both states, coerce_bool string forms ("false"/"False"/"0"/"no" plus the true forms) and unset cases (None and the empty string an unset action input forwards), severity/action proving verified → critical/error (blocks) and unverified → low/ignore (does not), and both fail-closed paths.

Two existing tests asserted the old behavior and are updated rather than worked around:

• tests/test_changed_files_scope.py:890 asserted --no-verification was in the command
• tests/test_trufflehog_excludes.py:474 asserted a failed run returns {}; it now expects the failure to surface, still asserting temp-file cleanup

I confirmed the coerce_bool test is not vacuous: it fails against the truthiness read and passes with the fix.

⚠️ Behavioral change — needs reviewer eyes

Three user-visible changes on the default path, all documented explicitly in the CHANGELOG and both user-facing docs pages:

1. Runs will start failing that previously passed. Verified secrets are now critical and blocking where they were previously downgraded to low. Not new detections — those secrets were always present, just never surfaced as blocking. Separately, a TruffleHog run that exits non-zero now fails instead of exiting green.
2. TruffleHog now makes outbound network requests, validating candidates against third-party endpoints (AWS, GitHub, Slack, …). Runs that previously scanned fully offline no longer do.
3. Air-gapped/proxied runners will under-report. Unreachable validation endpoints yield unknown, which --results=verified drops — so a no-egress scanner reports zero findings rather than failing loudly. Documented workaround: set trufflehog_show_unverified: true so unknown results still surface as low severity. This is the one silent-failure mode left in the change; the AC specified --results=verified explicitly, so I implemented that rather than quietly adding unknown. Worth a second opinion.

Also for a reviewer's call

is_enabled() reads secret_scanning_enabled with raw truthiness, as do scan_all and changed_files_scope_requested elsewhere in this connector. Same latent coerce_bool hazard as the Bugbot finding above, but fixing them changes behavior beyond this PR's scope — and they err toward enabling scanning rather than under-reporting secrets, so it's the less dangerous direction. Happy to do a follow-up sweep of bool reads across all connectors if you want it tracked separately.

Suggested version: 3.2.0 (minor), matching the precedent set by 3.1.0, which shipped a fail-the-run behavioral change as a minor. No version files are bumped here — CHANGELOG entries sit under [Unreleased] per the usual release-prep flow.

Docs

Corrected docs/parameters.md (which claimed "by default only verified secrets are shown" — false while --no-verification was in use), the input reference and example in docs/github-action.md, and the trufflehog_show_unverified descriptions in action.yml / connectors.yaml. Added egress warnings to both docs pages.

Drive-by: docs/parameters.md documented the JSON config key as show_unverified, which does not exist — a customer copying that example would get no effect. Now trufflehog_show_unverified. (Neighboring secrets_enabled looks similarly suspect but is referenced in github_pr_notifier.py:493, so I left it alone.)

Fixes CE-351
Fixes CE-347

🤖 Generated with Claude Code

---

Note

High Risk
Behavioral changes to secret scanning affect CI gating, introduce live verification network calls, and can silently under-report on restricted egress; misconfiguration now fails runs instead of passing quietly.

Overview
Fixes TruffleHog secret scanning so verification always runs and trufflehog_show_unverified only widens reported result types (--results=verified vs --results=verified,unverified,unknown), removing the old --no-verification path that made every finding unverified/low and non-blocking on the default setting. The connector now always passes --include-detectors=all, reads the flag via coerce_bool (so dashboard string "false" is not truthy), and fails the run on non-zero TruffleHog exit or a missing binary instead of returning an empty “clean” result.

Upgrade impact: verified secrets can start blocking runs that previously passed; scans require network egress for verification; air-gapped runners may see zero findings unless trufflehog_show_unverified is enabled. CHANGELOG, action inputs, and docs are updated accordingly, with new regression tests in tests/test_trufflehog_verification.py.

^{Reviewed by Cursor Bugbot for commit 64cf028. Configure here.}

[lelia]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 64cf028. Configure here.}