Skip to content

feat: plain English control explanations#44

Open
aanishs wants to merge 3 commits intomainfrom
feat/plain-english-explanations
Open

feat: plain English control explanations#44
aanishs wants to merge 3 commits intomainfrom
feat/plain-english-explanations

Conversation

@aanishs
Copy link
Copy Markdown
Owner

@aanishs aanishs commented Apr 4, 2026

Summary

  • New nist/plain-english.json with human-readable descriptions and why_it_matters for all HIPAA-mapped NIST controls
  • New /comply-explain skill: look up any control by NIST ID, HIPAA CFR section, or keyword
  • Enriched comply-db control --json output with plain English context, verification info, and related controls
  • Updated /comply-assess and /comply-auto to show why_it_matters during interviews
  • Updated architecture and skill-validation tests for 14 skills

Test plan

  • bun test test/plain-english.test.ts
  • bun test test/bin-smoke.test.ts
  • bun test test/architecture.test.ts
  • bun test test/skill-validation.test.ts
  • bin/comply-db control SC-28 --json returns plain_english and why_it_matters fields

🤖 Generated with Claude Code

Add plain-english.json with human-readable descriptions and why_it_matters
for all HIPAA-mapped NIST controls. New /comply-explain skill looks up any
control by ID, CFR section, or keyword. Enriches comply-db control --json
with plain English context. Updates comply-assess and comply-auto to show
why_it_matters during interviews.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Apr 4, 2026

Greptile Summary

This PR adds plain-English explanations for all 59 HIPAA-mapped NIST 800-53 controls and surfaces them through a new /comply-explain skill, enriched comply-db control --json output, and updated /comply-assess and /comply-auto interview flows. It also introduces bin/comply-check — a new PR-level compliance gate that scans git diffs for PHI patterns and BAA vendor dependencies — along with comply-db journey and comply-db config subcommands for milestone tracking and user preferences.

Key changes:

  • nist/plain-english.json: 59 plain-English descriptions with why_it_matters context, fully tested for coverage against hipaa-filter.json with no orphaned entries
  • bin/comply-db: buildControlData produces a richly enriched JSON payload from the NIST catalog, tool-bindings, and plain-english data; journey subcommand manages a 6-milestone compliance checklist with correct SKIPPED/COMPLETE distinction; config subcommand validates known keys and rejects unknown ones — all three issues flagged in the prior review are addressed in this PR
  • bin/comply-check: New CI gate binary that extracts code_grep patterns from checks-registry.ts at runtime and scans added lines for PHI leakage, insecure patterns, and newly-added BAA-required dependencies; has one P1 finding (git failure exits 0)
  • Tests: Both template and generated SKILL.md counts are consistently updated to 16; new smoke tests cover journey, config, and explain subcommands across normal and error paths"

Confidence Score: 4/5

Safe to merge after addressing the git-failure silent-pass in comply-check; existing functionality is not regressed

One P1 finding remains: the new comply-check CI gate exits 0 when git diff fails rather than exiting 2, which would produce false-clean results in CI pipelines. The two P2 findings (fragile regex parsing, dead file.content read) do not block merge. The comply-db changes are well-tested and all three concerns from the prior review thread are resolved in this PR.

bin/comply-check — the git exit-code check (P1) and optional refactor of loadCodeChecks (P2)

Important Files Changed

Filename Overview
bin/comply-check New CI compliance gate; has a P1 git-failure silent-pass bug and fragile runtime TypeScript source parsing via regex
bin/comply-db Adds buildControlData enrichment, journey milestones, config preferences, and CFR resolution; all previously flagged thread issues resolved
nist/plain-english.json 59 plain-English descriptions covering all HIPAA-mapped NIST controls; well-structured with nist_name, plain_english, and why_it_matters fields
nist/baa-vendors.json New vendor BAA list for 13 known services with correct baa_required flags; straightforward data file
test/plain-english.test.ts Thorough: full-coverage check, orphan detection, required-fields validation, and NIST catalog link validation
test/bin-smoke.test.ts New smoke tests for journey, config, and explain subcommands with good normal and error path coverage
test/architecture.test.ts Skill count updated from 13 to 16 with correct new skills listed; straightforward maintenance update
test/skill-validation.test.ts Template and generated counts both updated to 16; counts are consistent
skills/comply-explain/SKILL.md New skill for plain-English control lookup by NIST ID, CFR section, or keyword; well-structured workflow

Sequence Diagram

sequenceDiagram
    actor User
    participant Skill as /comply-explain skill
    participant DB as comply-db control --json
    participant PE as nist/plain-english.json
    participant NIST as NIST catalog
    participant TB as nist/tool-bindings.json
    participant HF as nist/hipaa-filter.json
    participant SQLite as SQLite DB

    User->>Skill: /comply-explain SC-28 (or CFR or keyword)
    Skill->>DB: comply-db control SC-28 --json
    DB->>NIST: findControlInCatalog(SC-28)
    NIST-->>DB: control prose, guidance, related links
    DB->>PE: loadPlainEnglish(SC-28)
    PE-->>DB: plain_english, why_it_matters
    DB->>HF: reverse-map SC-28 → CFR sections
    HF-->>DB: [164.312(a)(2)(iv)]
    DB->>TB: normalizeVerification(SC-28)
    TB-->>DB: tools, checks, interview_only
    DB->>SQLite: status, evidence_count, vendor_registry
    SQLite-->>DB: status=pending, evidenceCount=0
    DB-->>Skill: enriched JSON payload
    Skill-->>User: plain-English explanation + stack context + sources
Loading

Comments Outside Diff (1)

  1. bin/comply-check, line 151-156 (link)

    git diff failure silently exits 0

    getDiffFiles() calls Bun.spawnSync for git diff but never checks nameProc.exitCode. When the command fails — for example, an invalid --diff ref, or running outside a git repository — stdout is empty, names becomes [], and the function returns an empty DiffFile[].

    main() then hits the files.length === 0 branch and exits 0 with "No changed files to check." A compliance gate that returns a clean result on infrastructure failure gives false confidence in CI.

    Add an exit code check after the spawnSync call:

Reviews (5): Last reviewed commit: "feat: persona-driven skill architecture ..." | Re-trigger Greptile

Comment thread bin/comply-db Outdated
Comment thread test/skill-validation.test.ts Outdated
Comment thread bin/comply-db
Comment on lines +560 to +585
const jsonMode = args.includes('--json');
const simpleMode = args.includes('--simple');

// CFR resolution: if input looks like a HIPAA CFR section, resolve to NIST control IDs
let controlIds: string[];
let resolvedFrom: string | null = null;
if (isCFRSection(inputId)) {
controlIds = resolveFromCFR(inputId);
if (controlIds.length === 0) {
console.error(`CFR section ${inputId} not found in hipaa-filter.json.`);
process.exit(1);
}
resolvedFrom = inputId;
} else {
controlIds = [inputId];
}

const db = openDb();
// If multiple controls resolved from CFR, output each
if (jsonMode && controlIds.length > 1) {
const results = controlIds.map(id => buildControlData(id, resolvedFrom, simpleMode));
console.log(JSON.stringify(results, null, 2));
return;
}

// Single control (or first from CFR resolution for text mode)
const oscalId = controlIds[0];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 buildControlData silently returns null plain-English fields for non-HIPAA controls

loadPlainEnglish only covers the 59 HIPAA-mapped controls defined in nist/plain-english.json. When --json is used with any non-HIPAA control (e.g., from a SOC 2 or ISO 27001 project), pe is null and both plain_english and why_it_matters are null in the output — with no indication to the caller that the data simply doesn't exist for this framework.

The _sources.plain_english field already captures 'not_found', but the comply-explain skill doc tells users to call comply-db control <ID> --json for any control, implying coverage is complete. Consider adding a plain_english_coverage note in the output (e.g., 'hipaa_only') so consumers of the JSON can distinguish "no data" from "data not available for this framework."

Comment thread bin/comply-db
Comment thread bin/comply-db
aanishs and others added 2 commits April 6, 2026 00:20
Fix journey skip/complete identical status, normalize control ID casing,
error on invalid ID in --json mode, extract findControlInCatalog helper
(DRY), merge dual DB connections, reject unknown config keys, split
getDbPath read/write, revert premature comply-auto deprecation, add
journey + config + invalid-ID tests, fix skill counts to 14, add
plain-english.test.ts to test script.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…w skills

- /comply v3.0: rewritten as journey engine with persona detection,
  Framework Advisor (3 questions), milestone routing, and persona-aware
  recommendations for founders, engineers, and compliance officers
- bin/comply-check: new PR gate CLI that checks diffs for PHI patterns,
  sensitive data, removed security controls, and BAA-requiring dependencies
- nist/baa-vendors.json: curated list of 13 services requiring BAAs
- /comply-policy: new skill for generating personalized policy documents
- /comply-deal: new skill for answering prospect security questionnaires
- /soc2 v3.0: 3 domain-specific questions (trust criteria, audit type, driver)
- /gdpr v3.0: 4 domain-specific questions (role, locations, lawful basis, DPO)
- /pci-dss v3.0: 3 domain-specific questions (card handling, CDE scope, SAQ)
- /comply-auto v3.0: deprecated, points to /comply Autopilot mode
- Updated all comply-auto references across skill templates
- Tests updated: 14 → 16 skills (added comply-deal, comply-policy)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant