Skip to content

fix: redact interpolated secrets from telemetry and experience files (plan 003)#71

Open
DavertMik wants to merge 4 commits into
mainfrom
fix/redact-interpolated-secrets
Open

fix: redact interpolated secrets from telemetry and experience files (plan 003)#71
DavertMik wants to merge 4 commits into
mainfrom
fix/redact-interpolated-secrets

Conversation

@DavertMik

Copy link
Copy Markdown
Contributor

Implements plan 003 (plans/003-redact-secrets-from-persistence-and-telemetry.md) — a P1 security fix.

Problem

Knowledge files interpolate ${env.APP_PASSWORD} (and other ${env.*} / ${config.*}) into cleartext at load time. Those resolved secrets then flow into two persisted/exported sinks:

  • Langfuse trace input (JSON.stringify(metadata.input) shipped to telemetry).
  • On-disk experience files — which are routinely git-committed (the repo ships example/experience/*).

Separately, ${config.*} walked the entire config object and stringified any leaf, so ${config.ai.apiKey} silently expanded the provider API key into every prompt and both sinks.

Fix

  • New src/utils/secrets.ts: a process-global secret registry (registerSecret), redactSecrets(text), isSecretName(name) (credential-name denylist), and clearRegisteredSecrets() for test isolation.
  • knowledge-tracker.ts: the env branch registers credential-named values (still returns cleartext — the model must type the real password), and the config branch blocks credential-named keys from interpolating at all (SEC-03).
  • observability.ts + experience-tracker.ts: run redactSecrets on the serialized telemetry string and on experience content before write. Both are single chokepoints.

Verification

  • bun test tests/unit/secrets.test.ts tests/unit/knowledge-tracker.test.ts → 15 pass (incl. ${config.ai.apiKey} blocked, and ${env.APP_PASSWORD} register→redact end-to-end)
  • bun test tests/unit → 736 pass, 0 fail; bun run lint clean

Deferred (intentional — see plan Maintenance notes)

Masking the credential in (a) the LLM request itself (needs CodeceptJS secret() + a prompt change), (b) the AI SDK's experimental_telemetry inputs, and (c) the Testomat.io reporter. This PR is not a complete secret-masking solution — it closes the two persisted/exported sinks we control.

🤖 Generated with Claude Code

Knowledge files interpolate ${env.SECRET} into cleartext at load time; those
resolved values flowed into two persisted/exported sinks — the Langfuse trace
input and on-disk (often git-committed) experience files. Separately,
${config.*} could expand credential leaves like ${config.ai.apiKey} into every
prompt and both sinks.

Add src/utils/secrets.ts (a process-global secret registry + redactSecrets).
The env interpolation branch registers credential-named values (still returning
cleartext, since the model must type the real password) and both sinks now run
redactSecrets on the serialized/content string. Credential-named ${config.*}
paths are blocked from interpolating at all.

Deferred (see plan 003 Maintenance notes): masking in the LLM request itself,
the AI SDK experimental_telemetry inputs, and the Testomat.io reporter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown

Explorbot Self-Regression

Commit febcb6b · run

Scenario Result Attempts Duration
basic (native) PASS 3/3 19m
experience: control OK — failed as expected 1/1 2m
experience: seeded PASS 2/3 2m

Attempt details

  • basic (native) attempt 1 — FAIL: login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=6, wellFormed=true, keywords=6/3); scenarios: PASS (tests=6/5, features=4/3); tests passed: FAIL (4 passed, 1 failed (reporter: 4 passed, 1 failed))
  • basic (native) attempt 2 — FAIL: login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=7, wellFormed=true, keywords=7/3); scenarios: PASS (tests=6/5, features=4/3); tests passed: FAIL (3 passed, 2 failed (reporter: 3 passed, 2 failed))
  • basic (native) attempt 3 — PASS: login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=6, wellFormed=true, keywords=7/3); scenarios: PASS (tests=6/5, features=4/3); tests passed: PASS (5 passed, 0 failed (reporter: 5 passed, 0 failed))
  • experience: control attempt 1 — PASS: control: OK — failed as expected (0 passed, 1 failed)
  • experience: seeded attempt 1 — FAIL: seeded: FAIL (0 passed, 1 failed)
  • experience: seeded attempt 2 — PASS: seeded: PASS (1 passed, 0 failed)

Session analysis — basic (native):

Session Analysis

The Issues page core functionality is working: filtering by keyword, status, and label all function correctly, and viewing issue details works as expected. Issue creation succeeds functionally but the success confirmation message is missing. A visual indicator mismatch exists for label filtering.

Coverage

  • Pages: /issues, /issues/<id>
  • Features: Create issue, keyword search, status filter, label filter, view issue detail

What works

  • Issue creationET-1 — issue appears in list and count increases
  • Keyword searchET-2 — filters correctly, URL reflects query
  • Status filterET-3 — only closed issues display, filter persists
  • Label filterET-4 — bug-labeled issues display correctly (2 found)
  • View issue detailET-5 — navigates to /issues/4, title displays

Defects

[Medium] Success message not displayed after issue creation

Affects: ET-1 Create a new issue with all fields
Reproduce:

  1. Fill title, description, priority, labels, assignees
  2. Click Create Issue
    Evidence: Issue appears in list with count increased to 6, but no success message displays

UX issues

  • Label filter visual indicator — dropdown shows "All labels" active instead of "bug" after filter applied (ET-4)

Execution Issues

  • ET-4 — click failures on dropdown toggle, worked around via direct link click
  • ET-2 — reload/persistence test skipped due to automation limitation (keyboard shortcuts unreliable)

DavertMik and others added 2 commits July 11, 2026 16:29
Per re-review: iterate registered secrets longest-first in redactSecrets so a
secret that is a substring of another no longer leaves a partial-secret fragment
(genuine hardening for a security fix). Group SECRET_NAME_TOKENS with the other
module consts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread src/utils/secrets.ts Outdated
@DenysKuchma DenysKuchma self-requested a review July 15, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants