diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 513b15246..6cb915190 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -2173,6 +2173,7 @@ The `suggest_improvement` MCP tool allows AI agents to report gaps or errors. | [`src/CodeIndex/Mcp/McpToolHandlers.cs`](src/CodeIndex/Mcp/McpToolHandlers.cs) | `ExecuteSuggestImprovement` handler | | [`src/CodeIndex/Mcp/McpToolDefinitions.cs`](src/CodeIndex/Mcp/McpToolDefinitions.cs) | Tool schema definition | | [`src/CodeIndex/Cli/SuggestionsCommandRunner.cs`](src/CodeIndex/Cli/SuggestionsCommandRunner.cs) | Local suggestion listing, audited lifecycle transitions, bounded atomic export, issue-draft generation, and open-issue duplicate preflight | +| [`src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs`](src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs) | Redacted full-text history matching plus count, summary, compact, and byte-bounded JSON projections | ### What is sent (when GitHub token is configured) @@ -2205,6 +2206,10 @@ Local suggestion records use the `status` lifecycle field instead of a binary su `SuggestionStore.TryTransitionStatus` is the atomic manual-transition boundary used by `suggestions update --status `. `submitted_pending_triage` is automatic-only. `open_in_upstream` and `resolved_in_upstream` require existing upstream evidence; `draft` requires the absence of upstream evidence; and `wont_fix`, `duplicate`, or `superseded` are local maintainer dispositions. Local dispositions suppress automatic duplicate resubmission without setting `AlreadySubmitted` or an upstream-submission response flag. Same-state transitions and transitions during an active submission reservation fail closed. The store rechecks the expected revision under its file lock, stamps the latest `previous_status`, UTC `status_changed_at`, bounded/redacted `status_changed_by`, and optional bounded/redacted `status_change_reason`, updates `resolved_at` for `resolved_in_upstream`, and recomputes `revision_hash`. Full audit values are redacted before a surrogate-safe final cap so a credential crossing the cap boundary cannot evade redaction. Content edits and lifecycle transitions are separate CLI operations so one audit event has one unambiguous meaning. +`suggestions list|export --query ` matches the NFKC-normalized query as an ordinal, case-insensitive substring against the redacted stable ID, sampled title, description, context, evidence paths, category, and language. Applying `SuggestionStore.RedactSensitiveText` before matching is a confidentiality contract: a caller cannot use zero-result/count differences to probe a value removed by redaction. Status, time, category, language, and agent filters run first; the text query follows; records are then ordered by descending `CreatedAt` and ordinal stable ID before offset/limit pagination. + +The structured history projections share one JSON envelope. `--count` and `--summary-only` summarize the complete filtered set rather than the requested page; they report zero pagination omissions and classify non-emitted records as projection omissions. Summary dimensions have fixed distinct-value caps (status 16, category 32, language 20) and expose their own omitted/truncated metadata. `--compact` emits only redacted bounded list fields. `--max-json-bytes` measures the serialized UTF-8 document plus its final platform newline, uses a logarithmic fitting-prefix search, and removes complete trailing result rows until the envelope fits. `total_count` remains authoritative, while `byte_limit_omitted_count`, `next_offset`, and recovery guidance describe byte truncation. Row-producing compact and byte-bounded modes reject `--limit 0` so every advertised continuation can progress. If the metadata-only envelope cannot fit, the runner writes no stdout JSON. These projections are local read-only operations and do not change the streaming store's retention or mutation contracts. + `suggestions export --format markdown|issue-drafts --output ` renders the bounded payload in memory, rejects payloads over 16 MiB before writing, and refuses the selected database or suggestion-store path. For existing files it compares filesystem identities as well as normalized path spelling, so symlinked parents, mount aliases, and hard links cannot bypass source protection. Existing destinations are rejected unless `--overwrite` is explicit. Publication uses a sibling temporary file, flushes its contents, and performs a same-filesystem no-overwrite move or atomic replacement; failed publication cleans the temporary file. The writer emits UTF-8 without a BOM, creates missing parent directories, and keeps JSON-format suggestion exports on stdout. Tests cover the store transition/revision contract, CLI validation and filtering, source-target alias rejection, no-overwrite race safety, replacement, and temporary-file cleanup. ### GitHub retry idempotency @@ -5872,6 +5877,7 @@ Unlist しても exact version restore は不可能になりません。これ | [`src/CodeIndex/Mcp/McpToolHandlers.cs`](src/CodeIndex/Mcp/McpToolHandlers.cs) | `ExecuteSuggestImprovement` ハンドラ | | [`src/CodeIndex/Mcp/McpToolDefinitions.cs`](src/CodeIndex/Mcp/McpToolDefinitions.cs) | ツールスキーマ定義 | | [`src/CodeIndex/Cli/SuggestionsCommandRunner.cs`](src/CodeIndex/Cli/SuggestionsCommandRunner.cs) | ローカル提案の一覧、監査付き lifecycle 遷移、上限付き原子的 export、issue draft 生成、open issue duplicate preflight | +| [`src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs`](src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs) | redaction 済みの履歴全文検索、および count、summary、compact、byte 上限付き JSON projection | ### 送信されるデータ（GitHubトークン設定時） @@ -5904,6 +5910,10 @@ suggestion sidecar は `DataDirectorySecurity.ResolveSensitiveSidecarDirectoryFo `SuggestionStore.TryTransitionStatus` は `suggestions update --status ` が使う原子的な手動遷移境界です。`submitted_pending_triage` は自動設定専用です。`open_in_upstream` と `resolved_in_upstream` には既存の upstream 根拠が必要で、`draft` には upstream 根拠がないことが必要です。`wont_fix`、`duplicate`、`superseded` はメンテナーによるローカルの判断です。ローカルの判断は重複提案の自動再送を抑止しますが、`AlreadySubmitted` や upstream 送信済み response flag は設定しません。同じ状態への遷移、および送信 reservation が active な間の遷移は fail closed になります。store は file lock 内で expected revision を再確認し、最新の `previous_status`、UTC の `status_changed_at`、上限・redaction 付きの `status_changed_by`、任意の上限・redaction 付き `status_change_reason` を stamp し、`resolved_in_upstream` では `resolved_at` を更新して、`revision_hash` を再計算します。監査値全体を redaction してから surrogate-safe な最終上限を適用するため、上限境界をまたぐ credential も redaction を回避できません。1件の監査 event の意味を曖昧にしないため、content 編集と lifecycle 遷移は別々の CLI 操作です。 +`suggestions list|export --query ` は、NFKC 正規化した query を、redaction 済みの stable ID、sampled title、description、context、evidence path、category、language に対して ordinal・大文字小文字を区別しない部分一致で照合します。照合前に `SuggestionStore.RedactSensitiveText` を適用することは confidentiality contract です。caller は 0 件結果や count の差を使って redaction により除去された値を探索できません。status、時刻、category、language、agent の filter を最初に適用し、次に text query、続いて `CreatedAt` 降順と ordinal stable ID の順で並べてから offset/limit pagination を行います。 + +履歴の structured projection は共通 JSON envelope を使います。`--count` と `--summary-only` は要求された page ではなく filter 後の全集合を要約し、pagination omission を 0、出力しない record を projection omission として報告します。summary dimension は distinct 値に固定上限（status 16、category 32、language 20）を持ち、それぞれ omitted/truncated metadata を公開します。`--compact` は redaction・上限付きの list field だけを出力します。`--max-json-bytes` は serialized UTF-8 document と末尾の platform newline を計測し、対数回の fitting-prefix search を使って envelope が収まるまで末尾の完全な result row だけを取り除きます。`total_count` は authoritative なまま、`byte_limit_omitted_count`、`next_offset`、recovery guidance が byte truncation を表します。row を返す compact / byte 上限付き mode は `--limit 0` を拒否し、公開する continuation が必ず進捗できるようにします。metadata-only envelope が収まらない場合、runner は stdout JSON を一切書きません。これらの projection は local read-only 操作で、streaming store の retention や mutation contract は変更しません。 + `suggestions export --format markdown|issue-drafts --output ` は上限付き payload をメモリ上で描画し、書き込み前に 16 MiB 超過を拒否し、選択中の database または suggestion-store path も拒否します。既存ファイルでは正規化した path 表記に加えて filesystem identity も比較するため、symlink 付き親 directory、mount alias、hard link で source 保護を迂回できません。既存の出力先は `--overwrite` を明示しない限り拒否します。公開処理は兄弟一時ファイルを使い、内容を flush してから同一 filesystem 上で no-overwrite move または原子的置換を行い、失敗時は一時ファイルを片付けます。writer は BOM なし UTF-8 を出力し、不足している親 directory を作成し、JSON 形式の suggestion export は stdout のままです。test は store の遷移・revision 契約、CLI validation と filtering、source target alias 拒否、no-overwrite の race safety、置換、一時ファイル cleanup を網羅します。 ### GitHub 再試行の冪等性 diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index d2def1215..e0aec44ff 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -654,6 +654,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding Local suggestion JSON storage: opaque/stable-ID allocation and legacy migration, all-editable-field revision conflicts, dedup hashing, submission-finalization races, persistence, corruption recovery, atomic writes. Keep suggestion-redaction cases table-driven with negative fixtures for structured PascalCase, snake_case, leading-underscore, and recipe identifiers plus positive fixtures for opaque mixed-character and known token formats; the persistence case should retain an identifier and redact a secret from the same context. - `DataDirectorySecurityTests.cs`, `ProgramCliTests.cs` suggestion-sidecar coverage Shared-temp database routing, owner-only directory/file modes, colocated private-directory behavior, and structured CLI filesystem failures. + Suggestion-history query coverage keeps NFKC/case-insensitive matching across every documented field, structured filters before deterministic pagination, aggregate omission reasons, compact redaction, progressing continuations, and whole-document UTF-8 byte budgets in the same production-runtime fixture. - `SourceCodeDetectorTests.cs` Source code leak prevention: allowed natural-language inputs vs rejected code blocks (fenced, indented, import runs, etc.). - `ConsoleUiTests.cs` @@ -1653,6 +1654,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" ローカル提案JSON蓄積: 不透明で安定した ID の割り当てと legacy migration、全編集対象 field の revision conflict、ハッシュ重複排除、submission finalization race、永続化、破損復旧、アトミック書き込み。提案 redaction のケースは table-driven に保ち、構造化された PascalCase、snake_case、先頭 underscore 付き、recipe 形式の識別子を negative fixture、不透明な混合文字列と既知の token 形式を positive fixture として含めてください。永続化ケースでは、同じ context 内の識別子を保持しつつ secret を伏字化することを確認します。 - `DataDirectorySecurityTests.cs`、`ProgramCliTests.cs` の suggestion-sidecar coverage shared-temp database routing、owner-only の directory / file mode、private directory での隣接配置、structured CLI filesystem failure。 + suggestion-history query coverage は、全 documented field の NFKC / case-insensitive 照合、決定的 pagination より先の structured filter、aggregate omission reason、compact redaction、進捗可能な continuation、document 全体の UTF-8 byte budget を同じ production-runtime fixture で維持します。 - `SourceCodeDetectorTests.cs` ソースコード漏洩防止: 許容される自然言語入力 vs 拒否されるコードブロック（フェンス、インデント、import連打等）。 - `ConsoleUiTests.cs` diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 1ffb985c6..4d88f8073 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -3445,6 +3445,10 @@ cdidx includes a `suggest_improvement` MCP tool for AI agents that hit gaps or b Use `cdidx suggestions list` to review recorded suggestions, `cdidx suggestions show ` to inspect one entry, and `cdidx suggestions export --format markdown` to share a filtered triage bundle with a team. Use `cdidx suggestions export --format issue-drafts --open-issues open-issues.json` to emit issue-ready drafts with title, labels, evidence paths, severity/confidence/evidence-count triage metadata, body text, and duplicate matches from an open-issues JSON preflight. Add `--duplicate-confidence low|medium|high` or `--duplicate-threshold <0..1>` when issue-draft exports need looser or stricter duplicate matching. The command reads the selected DB's colocated private suggestion store or its deterministic user-scoped shared-temp fallback (`.cdidx/suggestions-codeindex.json` by default), supports filters such as `--status`, `--language`, `--category`, `--since`, and `--agent`, and prints JSON with `--json` for scripts. By default, `suggestions list` and `suggestions export` emit every matching record in newest-first order; pass `--limit ` and `--offset ` to page or cap large stores. Exported JSON, markdown bundles, and issue-draft bodies cap long description/context/tool-invocation text with a `[truncated]` marker; use `cdidx suggestions show ` when you need the full local record body. Treat exported issue drafts as triage aids and review duplicate guidance plus current open issues before filing. +For full-text triage, add `--query ` to `suggestions list` or `suggestions export`. It performs a Unicode NFKC-normalized, case-insensitive substring search over the redacted stable ID, sampled title, description, context, evidence paths, category, and language; normalized queries longer than 1000 characters are rejected. Redaction happens before matching, so a secret removed from persisted output cannot be discovered by querying for its original value. Exact filters and `--query` are applied before deterministic newest-first ordering (`created_at`, then stable ID) and before `--offset` / `--limit`. + +Large-history automation can select a bounded JSON projection. `--count` returns the authoritative filtered count (a scalar for human `suggestions list`, or a JSON envelope with `--json`); `--summary-only` returns bounded `by_status`, `by_category`, and `by_language` counts without record bodies; and `--compact` returns only `id`, bounded redacted `title`, `status`, and redacted `evidence_paths`. `--summary-only` and `--compact` imply JSON. Add `--max-json-bytes ` to cap the complete UTF-8 JSON document, including its final newline. When the limit removes rows, cdidx removes only whole trailing rows and reports authoritative `total_count`, `returned_count`, `byte_limit_omitted_count`, `truncated`, `next_offset`, and `recovery_guidance`; resume with the reported offset or increase the byte limit. A limit too small for the metadata-only envelope fails without emitting partial JSON. Count, summary, and compact are mutually exclusive, and structured projection flags on `suggestions export` require `--format json`. Row-producing compact and byte-bounded projections require a positive `--limit`; aggregate count and summary modes continue to ignore pagination. + Maintainers can make an explicit audited transition with `cdidx suggestions update --status wont_fix --actor --reason ""`. Manual targets are `draft`, `open_in_upstream`, `resolved_in_upstream`, `wont_fix`, `duplicate`, and `superseded`; `submitted_pending_triage` is reserved for successful GitHub submission. The two upstream states require an existing upstream URL or issue number, while returning to `draft` is allowed only when no upstream reference exists. Local `wont_fix`, `duplicate`, and `superseded` dispositions suppress automatic resubmission of the same suggestion but remain distinguishable from an actual upstream submission. A status transition cannot be combined with content edits, refuses a no-op transition, changes `revision_hash`, and records the latest `previous_status`, `status_changed_at`, `status_changed_by`, and optional `status_change_reason`. Audit text is redacted before its final length cap, and the actor defaults to `cdidx-cli` when omitted. Markdown and issue-draft exports can be published directly with `--output `, for example `cdidx suggestions export --format markdown --output suggestions.md`. File output is UTF-8 without a BOM, creates missing parent directories, and is capped at 16 MiB; use `--limit` and `--offset` to split larger exports. cdidx refuses an existing destination by default, rejects the selected database and suggestion-store paths including equivalent filesystem aliases, and uses a sibling temporary file plus a same-filesystem publish so a partial payload is never exposed. Pass `--overwrite` to replace an existing destination atomically. JSON-format suggestion exports remain stdout-only; issue-draft output files contain the same JSON object that would otherwise be printed. With `--json`, a successful file export writes a structured summary containing `status`, `format`, `count`, `output_path`, and `bytes` to stdout. @@ -6866,6 +6870,10 @@ cdidx には、AI エージェントがギャップや不具合に気づいた 記録済みの提案は `cdidx suggestions list` で確認し、`cdidx suggestions show ` で1件を詳細表示し、`cdidx suggestions export --format markdown` でチーム triage 用に共有できます。`cdidx suggestions export --format issue-drafts --open-issues open-issues.json` は、title、labels、evidence paths、severity / confidence / evidence-count の triage metadata、body text、open issue JSON との重複候補を含む Issue 作成用 draft を出力します。issue-draft export で重複一致を緩く、または厳しくしたい場合は `--duplicate-confidence low|medium|high` または `--duplicate-threshold <0..1>` を追加します。このコマンドは選択した DB に隣接する private な提案ストア、または deterministic な user-scoped shared-temp fallback（既定は `.cdidx/suggestions-codeindex.json`）を読み、`--status`、`--language`、`--category`、`--since`、`--agent` で絞り込めます。スクリプト向けには `--json` を使います。既定では `suggestions list` と `suggestions export` は一致した全レコードを新しい順に出力します。大きなストアでは `--limit ` と `--offset ` でページングまたは出力上限を指定できます。export JSON、markdown bundle、issue draft body は長い description / context / tool-invocation text を `[truncated]` marker 付きで制限します。ローカルレコード本文をすべて確認する場合は `cdidx suggestions show ` を使ってください。出力された issue draft は triage aid として扱い、起票前に duplicate guidance と現在の open issue を確認してください。 +全文 triage では、`suggestions list` または `suggestions export` に `--query ` を追加します。redaction 済みの stable ID、sampled title、description、context、evidence path、category、language に対して Unicode NFKC 正規化と大文字小文字を区別しない部分一致検索を行い、正規化後に 1000 文字を超える query は拒否します。照合前に redaction するため、出力から除去された secret を元の値で検索して発見することはできません。厳密 filter と `--query` は、決定的な新しい順の並び（`created_at`、次に stable ID）および `--offset` / `--limit` より先に適用されます。 + +大きな履歴を扱う automation では、上限付き JSON projection を選べます。`--count` は filter 後の authoritative な件数を返します（人間向け `suggestions list` では scalar、`--json` 併用時は JSON envelope）。`--summary-only` は record 本文を含めず、上限付きの `by_status`、`by_category`、`by_language` 件数を返します。`--compact` は `id`、上限・redaction 済み `title`、`status`、redaction 済み `evidence_paths` だけを返します。`--summary-only` と `--compact` は JSON を暗黙に有効化します。`--max-json-bytes ` は末尾改行を含む UTF-8 JSON document 全体を制限します。上限によって row が省略される場合、cdidx は末尾の完全な row だけを取り除き、authoritative な `total_count`、`returned_count`、`byte_limit_omitted_count`、`truncated`、`next_offset`、`recovery_guidance` を返します。報告された offset から再開するか byte 上限を増やしてください。metadata-only envelope にも足りない上限では、partial JSON を出さず失敗します。count、summary、compact は互いに排他的で、`suggestions export` の structured projection flag は `--format json` を必要とします。row を返す compact / byte 上限付き projection の `--limit` は正数でなければならず、aggregate の count / summary mode は pagination を引き続き無視します。 + メンテナーは `cdidx suggestions update --status wont_fix --actor --reason ""` で、監査情報付きの明示的な状態遷移を実行できます。手動で指定できる遷移先は `draft`、`open_in_upstream`、`resolved_in_upstream`、`wont_fix`、`duplicate`、`superseded` です。`submitted_pending_triage` は GitHub 送信成功時だけ自動設定されます。upstream の2状態には既存の upstream URL または Issue 番号が必要で、`draft` に戻せるのは upstream 参照がない場合だけです。ローカルの `wont_fix`、`duplicate`、`superseded` は同じ提案の自動再送を抑止しますが、実際の upstream 送信済み状態とは区別されます。状態遷移は content 編集と同時指定できず、同じ状態への遷移を拒否し、`revision_hash` を更新して、最新の `previous_status`、`status_changed_at`、`status_changed_by`、任意の `status_change_reason` を記録します。監査テキストは最終的な長さ制限より前に redaction され、`--actor` を省略した場合は `cdidx-cli` です。 Markdown と issue draft は `--output ` で直接ファイルへ公開できます。たとえば `cdidx suggestions export --format markdown --output suggestions.md` です。ファイル出力は BOM なし UTF-8 で、不足している親ディレクトリを作成し、16 MiB に制限されます。より大きい export は `--limit` と `--offset` で分割してください。既存の出力先は既定で拒否し、同じファイルを指す filesystem alias を含め、選択中の database または suggestion store への出力も拒否します。兄弟一時ファイルから同一 filesystem 上で公開するため、不完全な payload は見えません。既存ファイルを原子的に置換する場合だけ `--overwrite` を指定します。JSON 形式の suggestion export は従来どおり stdout 専用で、issue-draft 出力ファイルには stdout に出す場合と同じ JSON object が入ります。`--json` を指定したファイル出力の成功時は、`status`、`format`、`count`、`output_path`、`bytes` を含む構造化 summary を stdout に出します。 diff --git a/changelog.d/unreleased/5061.added.md b/changelog.d/unreleased/5061.added.md new file mode 100644 index 000000000..408743640 --- /dev/null +++ b/changelog.d/unreleased/5061.added.md @@ -0,0 +1,28 @@ +--- +category: added +issues: + - 5061 +affected: + - src/CodeIndex/Cli/SuggestionsCommandRunner.cs + - src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs + - src/CodeIndex/Cli/SuggestionStore.cs + - src/CodeIndex/Cli/CliFlagSchema.cs + - src/CodeIndex/Cli/CliContractManifest.cs + - src/CodeIndex/Cli/ConsoleUi.cs + - tests/CodeIndex.Tests/ProgramCliTests.cs + - tests/CodeIndex.Tests/CliFlagSchemaTests.cs + - tests/CodeIndex.Tests/ConsoleUiTests.cs + - tests/CodeIndex.Tests/JsonOutputSnapshotTests.cs + - tests/CodeIndex.Tests/golden/suggestions-compact.json + - USER_GUIDE.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **Suggestion history now supports bounded full-text triage (#5061; builds on #1876 and #1877)** — `suggestions list` and JSON export accept redaction-safe `--query`, authoritative `--count`, bounded `--summary-only` and `--compact` projections, and `--max-json-bytes` with whole-row truncation and deterministic offset recovery. + +## 日本語 + +- **提案履歴で上限付きの全文 triage を行えるようになりました (#5061、#1876 と #1877 を基盤とする変更)** — `suggestions list` と JSON export は、redaction-safe な `--query`、authoritative な `--count`、上限付きの `--summary-only` / `--compact` projection、および完全な row 単位の truncate と決定的な offset recovery を行う `--max-json-bytes` に対応します。 diff --git a/src/CodeIndex/Cli/CliContractManifest.cs b/src/CodeIndex/Cli/CliContractManifest.cs index e8533a19e..7bd35639c 100644 --- a/src/CodeIndex/Cli/CliContractManifest.cs +++ b/src/CodeIndex/Cli/CliContractManifest.cs @@ -76,6 +76,7 @@ internal static class CliContractManifest new("references", "references.json"), new("impact", "impact.json"), new("excerpt", "excerpt.json"), + new("suggestions-compact", "suggestions-compact.json"), ]; private static IReadOnlyList LoadCliJsonRootTypes() => diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 7b2e9f01c..620ec632d 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -205,7 +205,7 @@ public static bool HasAuthoritativeHelpOptions(string command) => private static readonly string[] QueryCommands = [ "search", "recipes", "definition", "goto", "references", "callers", "callees", - "symbols", "files", "find", "inspect", "impact", + "symbols", "files", "find", "inspect", "impact", "suggestions", ]; private static readonly string[] LimitCapableCommands = @@ -235,7 +235,7 @@ public static bool HasAuthoritativeHelpOptions(string command) => private static readonly string[] CountCommands = [ "search", "definition", "references", "callers", "callees", "symbols", - "files", "find", "impact", "unused", "hotspots", "audit", "languages", + "files", "find", "impact", "unused", "hotspots", "audit", "languages", "suggestions", ]; private static readonly string[] StrictNotFoundCommands = [ @@ -267,7 +267,7 @@ public static bool HasAuthoritativeHelpOptions(string command) => private static readonly string[] ByteFormatCommands = ["files", "map"]; private static readonly string[] EntrypointConfidenceCommands = ["map"]; private static readonly string[] MapSectionCommands = ["map"]; - private static readonly string[] SummaryOnlyCommands = ["map", "search", "recipes", "audit", "symbols", "files", "deps", "unused", "hotspots", "languages"]; + private static readonly string[] SummaryOnlyCommands = ["map", "search", "recipes", "audit", "symbols", "files", "deps", "unused", "hotspots", "languages", "suggestions"]; private static readonly string[] DependencyCycleCommands = ["deps"]; private static readonly string[] LanguagesFilterCommands = ["languages"]; @@ -335,7 +335,7 @@ public static bool HasAuthoritativeHelpOptions(string command) => "validate", "deps", "impact", "unused", "hotspots", "suggestions", "languages", "db", "report", "upgrade", "doctor", "license", ]; - private static readonly string[] CompactJsonCommands = ["map", "inspect", "outline", "symbols", "unused", "status", "hotspots", "impact"]; + private static readonly string[] CompactJsonCommands = ["map", "inspect", "outline", "symbols", "unused", "status", "hotspots", "impact", "suggestions"]; private static readonly string[] FormatCommands = [ @@ -578,7 +578,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--env-domain", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact domain", PrimaryCommands = Set("doctor") }, new() { Name = "--env-category", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact category", PrimaryCommands = Set("doctor") }, new() { Name = "--env-sensitivity", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact sensitivity", PrimaryCommands = Set("doctor") }, - new() { Name = "--max-json-bytes", ValuePlaceholder = "", Description = "Bound emitted JSON bytes; bounded responses omit whole rows with recovery metadata, including schema-valid audit SARIF", PrimaryCommands = Set("search", "definition", "find", "status", "references", "callers", "callees", "excerpt", "inspect", "outline", "impact", "recipes", "audit", "map", "files", "symbols", "deps", "hotspots", "languages", "unused", "doctor") }, + new() { Name = "--max-json-bytes", ValuePlaceholder = "", Description = "Bound emitted JSON bytes; bounded responses omit whole rows with recovery metadata, including schema-valid audit SARIF", PrimaryCommands = Set("search", "definition", "find", "status", "references", "callers", "callees", "excerpt", "inspect", "outline", "impact", "recipes", "audit", "map", "files", "symbols", "deps", "hotspots", "languages", "unused", "doctor", "suggestions") }, new() { Name = "--next-steps", Description = "Search: print inspect/excerpt follow-up commands for top hits", PrimaryCommands = Set("search") }, new() { Name = "--exclude-comments", Description = "Search: suppress comment-only matches after origin classification", PrimaryCommands = Set("search") }, new() { Name = "--exclude-strings", Description = "Search: suppress string, regex, and help-text matches after origin classification", PrimaryCommands = Set("search") }, diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index 22a2db4d5..7a79af805 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -133,10 +133,10 @@ private static readonly (string Command, string Usage)[] CommandUsageLines = ("deps", "cdidx deps [--db ] [--json] [--format ] [--summary-only] [--max-json-bytes ] [--verbose] [--limit |--top ] [--cursor ] [--graph-budget ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--reverse] [--cycles] [--suppress-noise] [--symbol ] [--symbol-family ]"), ("unused", "cdidx unused [--db ] [--json] [--compact] [--summary-only] [--max-json-bytes ] [--verbose] [--limit |--top ] [--cursor ] [--audit-scope ] [--kind ] [--bucket ] [--min-confidence |--confidence ] [--actionable] [--all] [--visibility ] [--exclude-visibility ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--count] [--by-bucket]"), ("hotspots", "cdidx hotspots [--db ] [--json] [--format ] [--compact] [--fields ] [--cursor ] [--summary-only] [--max-json-bytes ] [--verbose] [--limit |--top ] [--kind ] [--visibility ] [--exclude-visibility ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--count] [--group-by ] [--group-by-name]"), - ("suggestions", "cdidx suggestions [list|show|export|add|update|delete] [id|description] [--db ] [--json] [--description ] [--context ] [--title ] [--evidence-path ] [--status ] [--actor ] [--reason ] [--language ] [--category ] [--since ] [--agent ] [--limit ] [--offset ] [--format ] [--output ] [--overwrite] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ]"), - ("suggestions-list", "cdidx suggestions list [--status ] [--language ] [--category ] [--since ] [--agent ] [--limit ] [--offset ] [--db ] [--json]"), + ("suggestions", "cdidx suggestions [list|show|export|add|update|delete] [id|description] [--db ] [--json] [--description ] [--context ] [--title ] [--evidence-path ] [--status ] [--actor ] [--reason ] [--language ] [--category ] [--since ] [--agent ] [--query ] [--count|--summary-only|--compact] [--max-json-bytes ] [--limit ] [--offset ] [--format ] [--output ] [--overwrite] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ]"), + ("suggestions-list", "cdidx suggestions list [--status ] [--language ] [--category ] [--since ] [--agent ] [--query ] [--count|--summary-only|--compact] [--max-json-bytes ] [--limit ] [--offset ] [--db ] [--json]"), ("suggestions-show", "cdidx suggestions show [--status ] [--language ] [--category ] [--since ] [--agent ] [--db ] [--json]"), - ("suggestions-export", "cdidx suggestions export [--format ] [--status ] [--language ] [--category ] [--since ] [--agent ] [--limit ] [--offset ] [--output [--overwrite]] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ] [--db ] [--json]"), + ("suggestions-export", "cdidx suggestions export [--format ] [--status ] [--language ] [--category ] [--since ] [--agent ] [--query ] [--count|--summary-only|--compact] [--max-json-bytes ] [--limit ] [--offset ] [--output [--overwrite]] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ] [--db ] [--json]"), ("suggestions-add", "cdidx suggestions add |--description [--category ] [--language ] [--context ] [--title ] [--evidence-path ] [--agent ] [--db ] [--json]"), ("suggestions-update", "cdidx suggestions update [--description ] [--context ] [--title ] [--evidence-path ] [--category ] [--language ] [--agent ] [--db ] [--json]"), ("suggestions-update", "cdidx suggestions update --status [--actor ] [--reason ] [--db ] [--json]"), @@ -244,6 +244,7 @@ private static readonly (string Command, string Note)[] CommandUsageNotes = ("db-restore-backups", "Example: `cdidx db restore-backups --restore --dry-run --json`."), ("suggestions-list", "Reads local suggestion records, applies filters first, then applies the non-negative --offset/--limit page without changing the store."), ("suggestions-list", "Example: `cdidx suggestions list --status draft --limit 20 --json`."), + ("suggestions-list", "--query searches the redacted title, description, context, evidence paths, category, language, and stable id before pagination. --count, --summary-only, --compact, and --max-json-bytes provide bounded JSON projections."), ("suggestions-show", "Requires a full suggestion id or an unambiguous id prefix; filters are applied before id resolution and the store is not changed."), ("suggestions-show", "Example: `cdidx suggestions show --json`."), ("suggestions-export", "--format defaults to json. --output is supported only for markdown and issue-drafts, and --overwrite replaces an existing export atomically; export never submits or opens GitHub issues."), diff --git a/src/CodeIndex/Cli/SuggestionStore.cs b/src/CodeIndex/Cli/SuggestionStore.cs index 13320444f..4e08bc6fc 100644 --- a/src/CodeIndex/Cli/SuggestionStore.cs +++ b/src/CodeIndex/Cli/SuggestionStore.cs @@ -693,6 +693,23 @@ public List Load(int skip, int take) return ReadFilteredUnlocked(_ => true, skip, take); } + /// + /// Stream a bounded store snapshot and materialize only records accepted by the caller. + /// The predicate observes normalized legacy/default fields. + /// 上限付き store snapshot をストリーミングし、caller が受理した record だけを実体化する。 + /// predicate には legacy/default field を正規化した record を渡す。 + /// + internal List LoadFiltered( + Func predicate, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(predicate); + return ReadFilteredUnlocked( + predicate, + normalizeDefaults: true, + cancellationToken: cancellationToken); + } + /// /// Mark a suggestion as submitted to GitHub by updating its URL and flag. /// The entire read-modify-write is protected by a file lock. @@ -933,14 +950,16 @@ private static double ResolveDedupThreshold() private List ReadFilteredUnlocked( Func predicate, int skip = 0, - int? take = null) + int? take = null, + bool normalizeDefaults = false, + CancellationToken cancellationToken = default) { if (!TryReadStoreSnapshot(out var snapshot)) return new List(); try { - return ReadFilteredSnapshotAsync(snapshot, predicate, skip, take, normalizeDefaults: false) + return ReadFilteredSnapshotAsync(snapshot, predicate, skip, take, normalizeDefaults, cancellationToken) .GetAwaiter() .GetResult(); } @@ -1004,12 +1023,15 @@ internal static async Task> ReadFilteredSnapshotAsync( if (recordsRead > MaxSuggestionStoreRecords) throw new JsonException($"Suggestion store contains more than {MaxSuggestionStoreRecords} records."); - if (record == null || !predicate(record)) + if (record == null) continue; if (normalizeDefaults) NormalizeRecordDefaults(record); + if (!predicate(record)) + continue; + if (skipped < skip) { skipped++; diff --git a/src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs b/src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs new file mode 100644 index 000000000..632a80c3e --- /dev/null +++ b/src/CodeIndex/Cli/SuggestionsCommandRunner.Query.cs @@ -0,0 +1,266 @@ +using System.Text; +using System.Text.Json; +using System.Text.Json.Nodes; +using CodeIndex.Database; +using CodeIndex.Models; + +namespace CodeIndex.Cli; + +internal static partial class SuggestionsCommandRunner +{ + private const int SuggestionSummaryStatusLimit = 16; + private const int SuggestionSummaryCategoryLimit = 32; + private const int SuggestionSummaryLanguageLimit = 20; + + private static bool MatchesQuery(SuggestionRecord record, string normalizedQuery) + { + if (normalizedQuery.Length == 0) + return true; + + if (ContainsSuggestionQuery(record.Id, normalizedQuery) + || ContainsSuggestionQuery(record.SampledTitle, normalizedQuery) + || ContainsSuggestionQuery(record.Description, normalizedQuery) + || ContainsSuggestionQuery(record.Context, normalizedQuery) + || ContainsSuggestionQuery(record.Category, normalizedQuery) + || ContainsSuggestionQuery(record.Language, normalizedQuery)) + { + return true; + } + + if (record.EvidencePaths == null) + return false; + + foreach (var evidencePath in record.EvidencePaths) + { + if (ContainsSuggestionQuery(evidencePath, normalizedQuery)) + return true; + } + + return false; + } + + private static bool ContainsSuggestionQuery(string? value, string normalizedQuery) + { + if (string.IsNullOrWhiteSpace(value)) + return false; + + var redacted = SuggestionStore.RedactSensitiveText(value, out _); + return NormalizeSuggestionQueryText(redacted).Contains(normalizedQuery, StringComparison.OrdinalIgnoreCase); + } + + private static string NormalizeSuggestionQueryText(string value) + => value.Normalize(NormalizationForm.FormKC).Trim(); + + private static int RunStructuredQueryOutput( + IReadOnlyList filteredRecords, + IReadOnlyList pageRecords, + Options options, + JsonSerializerOptions jsonOptions, + bool exportDetails) + { + var resultNodes = new JsonArray(); + if (!options.Count && !options.SummaryOnly) + { + foreach (var record in pageRecords) + { + resultNodes.Add(options.Compact + ? ToCompactQueryItem(record) + : SerializeStructuredQueryItem(record, jsonOptions, exportDetails)); + } + } + + var totalCount = filteredRecords.Count; + var offset = Math.Min(options.Offset, totalCount); + var aggregateMode = options.Count || options.SummaryOnly; + var pageCount = aggregateMode ? 0 : pageRecords.Count; + var payload = new JsonObject + { + ["api_version"] = JsonOutputContract.ApiVersion, + ["mode"] = options.Count ? "count" : options.SummaryOnly ? "summary" : options.Compact ? "compact" : "full", + ["query"] = RedactSuggestionOutputValue(options.Query), + ["total_count"] = totalCount, + ["total_count_authoritative"] = true, + ["returned_count"] = resultNodes.Count, + ["offset"] = offset, + ["omitted_count"] = Math.Max(0, totalCount - resultNodes.Count), + ["pagination_omitted_count"] = aggregateMode ? 0 : Math.Max(0, totalCount - pageCount), + ["byte_limit_omitted_count"] = 0, + ["projection_omitted_count"] = options.Count || options.SummaryOnly ? totalCount : 0, + ["truncated"] = false, + ["has_more"] = false, + ["next_offset"] = null, + ["results"] = resultNodes, + }; + + if (options.Count) + payload["count"] = totalCount; + if (options.SummaryOnly) + payload["summary"] = BuildSuggestionSummary(filteredRecords); + if (options.MaxJsonBytes != null) + payload["output_byte_limit"] = options.MaxJsonBytes.Value; + + return WriteBoundedStructuredQueryPayload(payload, resultNodes, totalCount, offset, pageCount, options, jsonOptions); + } + + private static JsonNode SerializeStructuredQueryItem( + SuggestionRecord record, + JsonSerializerOptions jsonOptions, + bool exportDetails) + { + var context = CliJsonSerializerContextFactory.Create(jsonOptions); + return exportDetails + ? JsonSerializer.SerializeToNode(ToExportDetail(record), context.SuggestionDetailJsonResult)! + : JsonSerializer.SerializeToNode(ToListItem(record), context.SuggestionListItemJsonResult)!; + } + + private static JsonObject ToCompactQueryItem(SuggestionRecord record) + { + var redactedTitle = RedactSuggestionOutputValue(record.SampledTitle ?? record.Description) ?? string.Empty; + var evidencePaths = new JsonArray(); + foreach (var evidencePath in NormalizeEvidencePaths(record).Take(SuggestionEvidencePaths.MaxCount)) + { + var redactedPath = RedactSuggestionOutputValue(evidencePath); + if (redactedPath != null) + evidencePaths.Add(redactedPath); + } + + return new JsonObject + { + ["id"] = record.Id, + ["title"] = FormatTitle(redactedTitle, 120), + ["status"] = GetStatus(record), + ["evidence_paths"] = evidencePaths, + }; + } + + private static JsonObject BuildSuggestionSummary(IReadOnlyList records) + { + return new JsonObject + { + ["by_status"] = BuildSuggestionCountSummary(records.Select(GetStatus), SuggestionSummaryStatusLimit), + ["by_category"] = BuildSuggestionCountSummary( + records.Select(record => RedactSuggestionOutputValue(record.Category) ?? "unknown"), + SuggestionSummaryCategoryLimit), + ["by_language"] = BuildSuggestionCountSummary( + records.Select(record => string.IsNullOrWhiteSpace(record.Language) + ? "unknown" + : RedactSuggestionOutputValue(record.Language) ?? "unknown"), + SuggestionSummaryLanguageLimit), + }; + } + + private static JsonObject BuildSuggestionCountSummary(IEnumerable values, int limit) + { + var groups = values + .GroupBy(value => value, StringComparer.OrdinalIgnoreCase) + .Select(group => (Name: group.Key, Count: group.Count())) + .OrderByDescending(group => group.Count) + .ThenBy(group => group.Name, StringComparer.Ordinal) + .ToList(); + var counts = new JsonObject(); + foreach (var group in groups.Take(limit)) + counts[group.Name] = group.Count; + + return new JsonObject + { + ["distinct_count"] = groups.Count, + ["returned_distinct_count"] = counts.Count, + ["omitted_distinct_count"] = Math.Max(0, groups.Count - counts.Count), + ["truncated"] = groups.Count > counts.Count, + ["counts"] = counts, + }; + } + + private static int WriteBoundedStructuredQueryPayload( + JsonObject payload, + JsonArray results, + int totalCount, + int offset, + int pageCount, + Options options, + JsonSerializerOptions jsonOptions) + { + UpdateStructuredQueryPayloadMetadata(payload, results.Count, totalCount, offset, pageCount, options); + var json = payload.ToJsonString(jsonOptions); + if (options.MaxJsonBytes == null || GetTerminatedUtf8ByteCount(json) <= options.MaxJsonBytes.Value) + { + CommandOutputWriter.WriteRawJson(json); + return CommandExitCodes.Success; + } + + var resultRows = results.Select(static node => node!).ToArray(); + ResizeStructuredQueryResults(results, resultRows, 0); + UpdateStructuredQueryPayloadMetadata(payload, 0, totalCount, offset, pageCount, options); + var emptyJson = payload.ToJsonString(jsonOptions); + var emptyByteCount = GetTerminatedUtf8ByteCount(emptyJson); + if (emptyByteCount > options.MaxJsonBytes.Value) + { + return WriteUsageError( + $"--max-json-bytes {options.MaxJsonBytes.Value} is too small for the suggestion metadata envelope; at least {emptyByteCount} bytes are required.", + json: false, + jsonOptions, + "Increase --max-json-bytes; no partial JSON was emitted."); + } + + var fittingCount = 0; + var failingCount = resultRows.Length; + while (fittingCount + 1 < failingCount) + { + var candidateCount = fittingCount + ((failingCount - fittingCount) / 2); + ResizeStructuredQueryResults(results, resultRows, candidateCount); + UpdateStructuredQueryPayloadMetadata(payload, candidateCount, totalCount, offset, pageCount, options); + var candidateJson = payload.ToJsonString(jsonOptions); + if (GetTerminatedUtf8ByteCount(candidateJson) <= options.MaxJsonBytes.Value) + fittingCount = candidateCount; + else + failingCount = candidateCount; + } + + ResizeStructuredQueryResults(results, resultRows, fittingCount); + UpdateStructuredQueryPayloadMetadata(payload, fittingCount, totalCount, offset, pageCount, options); + json = payload.ToJsonString(jsonOptions); + CommandOutputWriter.WriteRawJson(json); + return CommandExitCodes.Success; + } + + private static void UpdateStructuredQueryPayloadMetadata( + JsonObject payload, + int returnedCount, + int totalCount, + int offset, + int pageCount, + Options options) + { + var byteLimitOmittedCount = Math.Max(0, pageCount - returnedCount); + var nextOffset = offset + returnedCount; + var hasMore = !options.Count && !options.SummaryOnly && nextOffset < totalCount; + payload["returned_count"] = returnedCount; + payload["omitted_count"] = Math.Max(0, totalCount - returnedCount); + payload["byte_limit_omitted_count"] = byteLimitOmittedCount; + payload["truncated"] = byteLimitOmittedCount > 0; + payload["has_more"] = hasMore; + payload["next_offset"] = hasMore && returnedCount > 0 ? nextOffset : null; + if (byteLimitOmittedCount > 0) + { + payload["recovery_guidance"] = returnedCount > 0 + ? $"Increase --max-json-bytes or resume with --offset {nextOffset}." + : "Increase --max-json-bytes; the first remaining row does not fit the current byte limit."; + } + else + payload.Remove("recovery_guidance"); + } + + private static void ResizeStructuredQueryResults( + JsonArray results, + IReadOnlyList resultRows, + int count) + { + while (results.Count > count) + results.RemoveAt(results.Count - 1); + while (results.Count < count) + results.Add(resultRows[results.Count]); + } + + private static int GetTerminatedUtf8ByteCount(string json) + => Encoding.UTF8.GetByteCount(json) + Encoding.UTF8.GetByteCount(Environment.NewLine); +} diff --git a/src/CodeIndex/Cli/SuggestionsCommandRunner.cs b/src/CodeIndex/Cli/SuggestionsCommandRunner.cs index 4c11d4226..769fa2067 100644 --- a/src/CodeIndex/Cli/SuggestionsCommandRunner.cs +++ b/src/CodeIndex/Cli/SuggestionsCommandRunner.cs @@ -9,9 +9,9 @@ namespace CodeIndex.Cli; -internal static class SuggestionsCommandRunner +internal static partial class SuggestionsCommandRunner { - private const string Usage = "Usage: cdidx suggestions [list|show|export|add|update|delete] [id|description] [--db ] [--json] [--description ] [--context ] [--title ] [--evidence-path ] [--status ] [--actor ] [--reason ] [--language ] [--category ] [--since ] [--agent ] [--limit ] [--offset ] [--format ] [--output ] [--overwrite] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ]"; + private const string Usage = "Usage: cdidx suggestions [list|show|export|add|update|delete] [id|description] [--db ] [--json] [--description ] [--context ] [--title ] [--evidence-path ] [--status ] [--actor ] [--reason ] [--language ] [--category ] [--since ] [--agent ] [--query ] [--count|--summary-only|--compact] [--max-json-bytes ] [--limit ] [--offset ] [--format ] [--output ] [--overwrite] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ]"; internal const int MaxOpenIssuesJsonBytes = IssueDuplicatePreflight.MaxOpenIssuesJsonBytes; internal const int MaxOpenIssuesJsonDepth = IssueDuplicatePreflight.MaxOpenIssuesJsonDepth; internal const int MaxSuggestionExportTextFieldLength = 4096; @@ -78,6 +78,29 @@ public static int Run( { return WriteUsageError(StripErrorPrefix(options.Error), options.Json, jsonOptions); } + if (options.Count && options.SummaryOnly) + return WriteUsageError("--count and --summary-only cannot be combined.", options.Json, jsonOptions); + if (options.Count && options.Compact) + return WriteUsageError("--count and --compact cannot be combined.", options.Json, jsonOptions); + if (options.SummaryOnly && options.Compact) + return WriteUsageError("--summary-only and --compact cannot be combined.", options.Json, jsonOptions); + if (options.Limit == 0 + && !options.Count + && !options.SummaryOnly + && (options.Compact || options.MaxJsonBytes != null)) + { + return WriteUsageError( + "--limit 0 cannot be used with --compact or --max-json-bytes because the structured page could not provide a progressing continuation offset.", + options.Json, + jsonOptions, + "Use a positive --limit, or remove the page projection options."); + } + if (options.HasHistoryQueryProjectionOptions && verb is not ("list" or "export")) + return WriteUsageError("--query, --count, --summary-only, --compact, and --max-json-bytes can only be used with `suggestions list` or `suggestions export`.", options.Json, jsonOptions); + if (verb == "export" + && options.HasStructuredProjectionOptions + && options.ExportFormat != "json") + return WriteUsageError("--count, --summary-only, --compact, and --max-json-bytes require `suggestions export --format json`.", options.Json, jsonOptions); if ((options.DuplicateConfidenceSpecified || options.DuplicateThresholdSpecified) && (verb != "export" || options.ExportFormat != "issue-drafts")) return WriteUsageError("--duplicate-confidence and --duplicate-threshold can only be used with `suggestions export --format issue-drafts`.", options.Json, jsonOptions); @@ -124,10 +147,17 @@ public static int Run( if (verb == "delete") return RunDelete(store, store.LoadAll(), options, jsonOptions); - records = ApplyFilters(store.LoadAll(), options) - .OrderByDescending(s => s.CreatedAt) - .ThenBy(s => s.Id, StringComparer.Ordinal) - .ToList(); + records = store.LoadFiltered(record => MatchesFilters(record, options), cancellationToken); + if (!options.Count && !options.SummaryOnly) + { + records.Sort(static (left, right) => + { + var createdAtComparison = right.CreatedAt.CompareTo(left.CreatedAt); + return createdAtComparison != 0 + ? createdAtComparison + : StringComparer.Ordinal.Compare(left.Id, right.Id); + }); + } } catch (Exception ex) when (IsSuggestionStoreFileSystemException(ex)) { @@ -139,10 +169,11 @@ public static int Run( return verb switch { - "list" => RunList(outputRecords, records.Count, options, jsonOptions), + "list" => RunList(outputRecords, records, options, jsonOptions), "show" => RunShow(records, options, jsonOptions), "export" => RunExport( outputRecords, + records, options, jsonOptions, cancellationToken, @@ -402,8 +433,21 @@ private static int RunAdd(SuggestionStore store, Options options, JsonSerializer return CommandExitCodes.Success; } - private static int RunList(List records, int totalCount, Options options, JsonSerializerOptions jsonOptions) + private static int RunList( + List records, + List filteredRecords, + Options options, + JsonSerializerOptions jsonOptions) { + var totalCount = filteredRecords.Count; + if (options.Count && !options.Json) + { + Console.WriteLine(totalCount.ToString(CultureInfo.InvariantCulture)); + return CommandExitCodes.Success; + } + if (options.HasStructuredProjectionOptions) + return RunStructuredQueryOutput(filteredRecords, records, options, jsonOptions, exportDetails: false); + if (options.Json) { var offset = Math.Min(options.Offset, totalCount); @@ -514,12 +558,16 @@ private static int RunShow(List records, Options options, Json private static int RunExport( List records, + List filteredRecords, Options options, JsonSerializerOptions jsonOptions, CancellationToken cancellationToken, string suggestionStorePath, string databasePath) { + if (options.HasStructuredProjectionOptions) + return RunStructuredQueryOutput(filteredRecords, records, options, jsonOptions, exportDetails: true); + if (options.ExportFormat == "markdown") { var markdown = FormatMarkdown(records); @@ -757,26 +805,27 @@ private static int WriteSuggestionStoreAccessError( errorCode: CommandErrorCodes.SuggestionStoreUnavailable, category: FileSystemBoundary.ClassifyProbeFailure(ex)); - private static IEnumerable ApplyFilters(IEnumerable records, Options options) + private static bool MatchesFilters(SuggestionRecord record, Options options) { - foreach (var record in records) - { - if (options.Status != "all" && !MatchesStatus(record, options.Status)) - continue; - if (options.Language != null && !string.Equals(record.Language, options.Language, StringComparison.OrdinalIgnoreCase)) - continue; - if (options.Category != null && !string.Equals(record.Category, options.Category, StringComparison.OrdinalIgnoreCase)) - continue; - if (options.Agent != null && !MatchesAgent(record, options.Agent)) - continue; - if (options.Since != null && new DateTimeOffset(DateTime.SpecifyKind(record.CreatedAt, DateTimeKind.Utc)) < options.Since.Value) - continue; - yield return record; - } + if (options.Status != "all" && !MatchesStatus(record, options.Status)) + return false; + if (options.Language != null && !string.Equals(record.Language, options.Language, StringComparison.OrdinalIgnoreCase)) + return false; + if (options.Category != null && !string.Equals(record.Category, options.Category, StringComparison.OrdinalIgnoreCase)) + return false; + if (options.Agent != null && !MatchesAgent(record, options.Agent)) + return false; + if (options.Since != null && new DateTimeOffset(DateTime.SpecifyKind(record.CreatedAt, DateTimeKind.Utc)) < options.Since.Value) + return false; + if (options.Query != null && !MatchesQuery(record, options.Query)) + return false; + return true; } private static List ApplyOutputPage(List records, Options options) { + if (options.Count || options.SummaryOnly) + return new List(); if (options.Offset == 0 && options.Limit == null) return records; @@ -879,7 +928,19 @@ private static bool MatchesAgent(SuggestionRecord record, string agent) private static string FormatTitle(string description, int maxLength) { var firstLine = description.Replace('\r', ' ').Replace('\n', ' ').Trim(); - return firstLine.Length <= maxLength ? firstLine : firstLine[..(maxLength - 1)] + "..."; + if (firstLine.Length <= maxLength) + return firstLine; + + var end = maxLength - 1; + if (end > 0 + && end < firstLine.Length + && char.IsHighSurrogate(firstLine[end - 1]) + && char.IsLowSurrogate(firstLine[end])) + { + end--; + } + + return firstLine[..end] + "..."; } private static SuggestionListItemJsonResult ToListItem(SuggestionRecord record) => new( @@ -1418,6 +1479,45 @@ bool TryReadSchemaValue(string option, out string value, out string? error) else options.Since = parsedSince; break; + case "--query": + if (!TryReadSchemaValue("--query", out var query, out var queryError)) + { + options.Error = queryError; + return options; + } + var normalizedQuery = NormalizeSuggestionQueryText(query); + if (normalizedQuery.Length == 0) + options.Error = "Error: --query must not be empty."; + else if (normalizedQuery.Length > QueryLimits.MaxQueryLength) + options.Error = $"Error: --query must be at most {QueryLimits.MaxQueryLength} characters."; + else + options.Query = normalizedQuery; + break; + case "--count": + options.Count = true; + break; + case "--summary-only": + options.SummaryOnly = true; + options.Json = true; + break; + case "--compact": + options.Compact = true; + options.Json = true; + break; + case "--max-json-bytes": + if (!TryReadSchemaValue("--max-json-bytes", out var maxJsonBytes, out var maxJsonBytesError)) + { + options.Error = maxJsonBytesError; + return options; + } + if (!TryParsePositiveInt("--max-json-bytes", maxJsonBytes, MaxSuggestionExportFileBytes, out var parsedMaxJsonBytes, out var parsedMaxJsonBytesError)) + { + options.Error = parsedMaxJsonBytesError; + return options; + } + options.MaxJsonBytes = parsedMaxJsonBytes; + options.Json = true; + break; case "--format": if (!TryReadSchemaValue("--format", out var format, out var formatError)) { @@ -1542,6 +1642,26 @@ private static bool TryParseNonNegativeInt(string option, string rawValue, out i return false; } + private static bool TryParsePositiveInt( + string option, + string rawValue, + int maximum, + out int value, + out string? error) + { + if (int.TryParse(rawValue, NumberStyles.None, CultureInfo.InvariantCulture, out value) + && value > 0 + && value <= maximum) + { + error = null; + return true; + } + + value = 0; + error = $"Error: {option} must be an integer from 1 to {maximum}."; + return false; + } + private static bool TryParseScoreThreshold(string option, string rawValue, out double value, out string? error) { if (double.TryParse(rawValue, NumberStyles.Float, CultureInfo.InvariantCulture, out value) && @@ -1616,11 +1736,18 @@ private sealed class Options public bool DuplicateConfidenceSpecified { get; set; } public bool DuplicateThresholdSpecified { get; set; } public DateTimeOffset? Since { get; set; } + public string? Query { get; set; } + public bool Count { get; set; } + public bool SummaryOnly { get; set; } + public bool Compact { get; set; } + public int? MaxJsonBytes { get; set; } public string? Error { get; set; } public bool HasPagination => Limit.HasValue || OffsetSpecified; public bool HasContentEditableFields => LanguageSpecified || CategorySpecified || DescriptionSpecified || ContextSpecified || TitleSpecified || EvidencePathsSpecified || AgentSpecified; public bool HasQueryOnlyOptions => HasQueryOnlyOptionsExceptStatus || StatusSpecified; - public bool HasQueryOnlyOptionsExceptStatus => HasPagination || Since != null || FormatSpecified || OutputPath != null || Overwrite || OpenIssuesPath != null || OpenIssuesRepository != null || IssueStateSpecified || DuplicateConfidenceSpecified || DuplicateThresholdSpecified; + public bool HasQueryOnlyOptionsExceptStatus => HasPagination || Since != null || HasHistoryQueryProjectionOptions || FormatSpecified || OutputPath != null || Overwrite || OpenIssuesPath != null || OpenIssuesRepository != null || IssueStateSpecified || DuplicateConfidenceSpecified || DuplicateThresholdSpecified; + public bool HasHistoryQueryProjectionOptions => Query != null || HasStructuredProjectionOptions; + public bool HasStructuredProjectionOptions => Count || SummaryOnly || Compact || MaxJsonBytes != null; } } diff --git a/tests/CodeIndex.Tests/CliFlagSchemaTests.cs b/tests/CodeIndex.Tests/CliFlagSchemaTests.cs index fa1ee2988..83c81e611 100644 --- a/tests/CodeIndex.Tests/CliFlagSchemaTests.cs +++ b/tests/CodeIndex.Tests/CliFlagSchemaTests.cs @@ -289,13 +289,20 @@ public void SuggestionsParserFlags_IncludeHiddenLangAlias_Issue4162() var accepted = CliFlagSchema.GetAcceptedFlagNamesForCommand("suggestions"); Assert.Contains("--language", accepted); Assert.Contains("--lang", accepted); + Assert.Contains("--query", accepted); + Assert.Contains("--max-json-bytes", accepted); var (withValues, flagOnly) = CliFlagSchema.GetParserFlagsPartitionedByValueBearing("suggestions"); Assert.Contains("--lang", withValues); Assert.Contains("--language", withValues); Assert.Contains("--description", withValues); Assert.Contains("--evidence-path", withValues); + Assert.Contains("--query", withValues); + Assert.Contains("--max-json-bytes", withValues); Assert.Contains("--json", flagOnly); + Assert.Contains("--count", flagOnly); + Assert.Contains("--summary-only", flagOnly); + Assert.Contains("--compact", flagOnly); Assert.DoesNotContain("--json", withValues); Assert.DoesNotContain(CliFlagSchema.GetCompletionFlagsForCommand("suggestions"), flag => flag.Name == "--lang"); @@ -303,12 +310,21 @@ public void SuggestionsParserFlags_IncludeHiddenLangAlias_Issue4162() var parse = typeof(SuggestionsCommandRunner).GetMethod("Parse", BindingFlags.NonPublic | BindingFlags.Static); Assert.NotNull(parse); - var parsed = parse!.Invoke(null, [new[] { "--lang=csharp", "--json" }]); + var parsed = parse!.Invoke(null, [new[] { "--lang=csharp", "--query=needle", "--compact", "--max-json-bytes=4096" }]); Assert.NotNull(parsed); Assert.Equal("csharp", parsed!.GetType().GetProperty("Language")!.GetValue(parsed)); + Assert.Equal("needle", parsed.GetType().GetProperty("Query")!.GetValue(parsed)); + Assert.Equal(true, parsed.GetType().GetProperty("Compact")!.GetValue(parsed)); + Assert.Equal(4096, parsed.GetType().GetProperty("MaxJsonBytes")!.GetValue(parsed)); Assert.Equal(true, parsed.GetType().GetProperty("Json")!.GetValue(parsed)); Assert.Null(parsed.GetType().GetProperty("Error")!.GetValue(parsed)); + var oversizedQuery = parse.Invoke(null, [new[] { $"--query={new string('q', QueryLimits.MaxQueryLength + 1)}" }]); + Assert.NotNull(oversizedQuery); + Assert.Equal( + $"Error: --query must be at most {QueryLimits.MaxQueryLength} characters.", + oversizedQuery!.GetType().GetProperty("Error")!.GetValue(oversizedQuery)); + var rejected = parse.Invoke(null, [new[] { "--json=true" }]); Assert.NotNull(rejected); Assert.Equal("Error: --json does not take a value.", rejected!.GetType().GetProperty("Error")!.GetValue(rejected)); diff --git a/tests/CodeIndex.Tests/ConsoleUiTests.cs b/tests/CodeIndex.Tests/ConsoleUiTests.cs index 492454124..f8e78ff60 100644 --- a/tests/CodeIndex.Tests/ConsoleUiTests.cs +++ b/tests/CodeIndex.Tests/ConsoleUiTests.cs @@ -1471,9 +1471,12 @@ public void CompletionRenderer_SuggestionsExposesOutputAcrossShells_Issue4719() { var flagSets = ExtractComparableSubcommandFlagSets("suggestions", "export"); - Assert.Contains("output", flagSets.Bash); - Assert.Contains("output", flagSets.Zsh); - Assert.Contains("output", flagSets.Fish); + foreach (var flag in new[] { "output", "query", "count", "summary-only", "compact", "max-json-bytes" }) + { + Assert.Contains(flag, flagSets.Bash); + Assert.Contains(flag, flagSets.Zsh); + Assert.Contains(flag, flagSets.Fish); + } var powerShell = ConsoleCompletionRenderer.GetCompletionScript("powershell"); var suggestionsBranch = ExtractBetween( @@ -1482,6 +1485,11 @@ public void CompletionRenderer_SuggestionsExposesOutputAcrossShells_Issue4719() ") }"); Assert.Contains("'--output'", suggestionsBranch, StringComparison.Ordinal); Assert.Contains("'-o'", suggestionsBranch, StringComparison.Ordinal); + Assert.Contains("'--query'", suggestionsBranch, StringComparison.Ordinal); + Assert.Contains("'--count'", suggestionsBranch, StringComparison.Ordinal); + Assert.Contains("'--summary-only'", suggestionsBranch, StringComparison.Ordinal); + Assert.Contains("'--compact'", suggestionsBranch, StringComparison.Ordinal); + Assert.Contains("'--max-json-bytes'", suggestionsBranch, StringComparison.Ordinal); } private static (SortedSet Bash, SortedSet Zsh, SortedSet Fish, string BashScript, string ZshScript, string FishScript) diff --git a/tests/CodeIndex.Tests/JsonOutputSnapshotTests.cs b/tests/CodeIndex.Tests/JsonOutputSnapshotTests.cs index 177ba8fba..448553ecc 100644 --- a/tests/CodeIndex.Tests/JsonOutputSnapshotTests.cs +++ b/tests/CodeIndex.Tests/JsonOutputSnapshotTests.cs @@ -1,7 +1,9 @@ using System.Text.Json; using System.Text.Json.Nodes; +using System.Text.Json.Serialization.Metadata; using CodeIndex.Cli; using CodeIndex.Database; +using CodeIndex.Models; namespace CodeIndex.Tests; @@ -22,6 +24,7 @@ public class JsonOutputSnapshotTests private readonly JsonSerializerOptions _jsonOptions = new() { PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower, + TypeInfoResolver = new DefaultJsonTypeInfoResolver(), }; private const string LibSource = @"namespace Demo; @@ -233,6 +236,52 @@ public void RunExcerpt_JsonOutput_MatchesGolden() } } + [Fact] + public void RunSuggestionsCompact_JsonOutput_MatchesGolden_Issue5061() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_snapshot_suggestions_compact"); + try + { + var cdidxDir = Path.Combine(projectRoot, ".cdidx"); + Directory.CreateDirectory(cdidxDir); + var dbPath = Path.Combine(cdidxDir, "codeindex.db"); + var titlePrefix = new string('a', 118); + var store = new SuggestionStore(cdidxDir); + Assert.True(store.TryAdd(new SuggestionRecord + { + Id = new string('1', 64), + Category = "output_format", + Language = "csharp", + Description = "Snapshot query description", + SampledTitle = titlePrefix + "😀suffix", + EvidencePaths = ["src/Snapshot.cs"], + CreatedAt = new DateTime(2026, 8, 11, 0, 0, 0, DateTimeKind.Utc), + })); + + var (exitCode, stdout, stderr) = CaptureConsole(() => SuggestionsCommandRunner.Run( + ["list", "--db", dbPath, "--query", "snapshot query", "--compact"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = JsonDocument.Parse(stdout); + var item = Assert.Single(document.RootElement.GetProperty("results").EnumerateArray()); + var compactTitle = item.GetProperty("title").GetString() + ?? throw new InvalidOperationException("Compact suggestion title was null."); + Assert.Equal(titlePrefix + "...", compactTitle); + Assert.DoesNotContain('\uFFFD', compactTitle); + + JsonOutputSnapshotHelper.AssertMatches( + "suggestions-compact.json", + stdout, + BuildPathReplacements(projectRoot)); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + private static void MarkGraphAndFoldReady(string dbPath) { using var db = new DbContext(DbOpenIntent.WriteIndex, dbPath); diff --git a/tests/CodeIndex.Tests/ProgramCliTests.cs b/tests/CodeIndex.Tests/ProgramCliTests.cs index 6712a9cac..0bbf14a13 100644 --- a/tests/CodeIndex.Tests/ProgramCliTests.cs +++ b/tests/CodeIndex.Tests/ProgramCliTests.cs @@ -1173,6 +1173,173 @@ public void Suggestions_ListJsonSupportsLimitAndOffset() Assert.Equal("Middle suggestion", item.GetProperty("title").GetString()); } + [ProductionRuntimeFact] + public void Suggestions_QuerySearchesRedactedHistoryBeforePagination_Issue5061() + { + using var fixture = SuggestionFixture.Create(); + var title = fixture.Add( + "title_query_category", + "title_query_language", + "Unrelated description", + submitted: false, + sampledTitle: "Ｆｕｌｌｗｉｄｔｈ　Ｔｉｔｌｅ"); + var description = fixture.Add("description_query_category", null, "Unique description needle 5061", submitted: false); + var context = fixture.Add( + "context_query_category", + null, + "Unrelated context description", + submitted: false, + context: "Unique context needle 5061"); + var evidence = fixture.Add( + "evidence_query_category", + null, + "Unrelated evidence description", + submitted: false, + evidencePaths: ["src/UniqueEvidenceNeedle5061.cs"]); + var category = fixture.Add("unique_category_needle_5061", null, "Unrelated category description", submitted: false); + var language = fixture.Add("language_query_category", "unique_language_needle_5061", "Unrelated language description", submitted: false); + + AssertQueryReturns("fullwidth title", title); + AssertQueryReturns("DESCRIPTION NEEDLE 5061", description); + AssertQueryReturns("context needle 5061", context); + AssertQueryReturns("uniqueevidenceneedle5061", evidence); + AssertQueryReturns("unique_category_needle_5061", category); + AssertQueryReturns("unique_language_needle_5061", language); + AssertQueryReturns(title.Hash[..16], title); + + var olderMatch = fixture.Add("output_format", "csharp", "Pagination needle 5061 older", submitted: false); + fixture.Add("output_format", "csharp", "Newest but unrelated", submitted: false); + fixture.Add("output_format", "csharp", "Pagination needle 5061 newer", submitted: false); + var (pageExitCode, pageStdout, pageStderr) = RunCliInSubprocess([ + "suggestions", "list", "--db", fixture.DbPath, + "--category", "output_format", "--language", "csharp", + "--query", "pagination NEEDLE 5061", "--limit", "1", "--offset", "1", "--json" + ]); + + Assert.Equal(CommandExitCodes.Success, pageExitCode); + Assert.Equal(string.Empty, pageStderr); + using var pageDoc = JsonDocument.Parse(pageStdout); + Assert.Equal(2, pageDoc.RootElement.GetProperty("total_count").GetInt32()); + var pageItem = Assert.Single(pageDoc.RootElement.GetProperty("results").EnumerateArray()); + Assert.Equal(olderMatch.Hash, pageItem.GetProperty("id").GetString()); + + void AssertQueryReturns(string query, SuggestionRecord expected) + { + var (exitCode, stdout, stderr) = RunCliInSubprocess([ + "suggestions", "list", "--db", fixture.DbPath, "--query", query, "--json" + ]); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.True(string.IsNullOrEmpty(stderr), stderr); + using var doc = JsonDocument.Parse(stdout); + Assert.Equal(1, doc.RootElement.GetProperty("total_count").GetInt32()); + var result = Assert.Single(doc.RootElement.GetProperty("results").EnumerateArray()); + Assert.Equal(expected.Hash, result.GetProperty("id").GetString()); + } + } + + [ProductionRuntimeFact] + public void Suggestions_QueryProjectionsAreBoundedAndRedacted_Issue5061() + { + using var fixture = SuggestionFixture.Create(); + const string rawSecret = "query_secret_5061"; + for (var i = 0; i < 24; i++) + { + fixture.Add( + i % 2 == 0 ? "output_format" : "language_support", + i % 3 == 0 ? "csharp" : "rust", + $"bulk-marker-5061 suggestion {i:D2} {new string((char)('a' + i % 26), 1024)}", + submitted: false, + sampledTitle: i == 0 ? $"api-token={rawSecret}" : $"Bulk suggestion {i:D2}", + evidencePaths: i == 0 ? [$"src/api-token={rawSecret}.cs"] : [$"src/Bulk{i:D2}.cs"]); + } + + var (countExitCode, countStdout, countStderr) = RunCliInSubprocess([ + "suggestions", "list", "--db", fixture.DbPath, "--query", "BULK-MARKER-5061", "--count", "--json" + ]); + var (summaryExitCode, summaryStdout, summaryStderr) = RunCliInSubprocess([ + "suggestions", "list", "--db", fixture.DbPath, "--query", "bulk-marker-5061", "--summary-only" + ]); + var (compactExitCode, compactStdout, compactStderr) = RunCliInSubprocess([ + "suggestions", "export", "--db", fixture.DbPath, "--format", "json", + "--query", "bulk-marker-5061", "--compact", "--limit", "24" + ]); + var fullCompactBytes = Encoding.UTF8.GetByteCount(compactStdout); + var byteBudget = fullCompactBytes - 256; + var (boundedExitCode, boundedStdout, boundedStderr) = RunCliInSubprocess([ + "suggestions", "export", "--db", fixture.DbPath, "--format", "json", + "--query", "bulk-marker-5061", "--compact", "--limit", "24", + "--max-json-bytes", byteBudget.ToString(CultureInfo.InvariantCulture) + ]); + var (secretExitCode, secretStdout, secretStderr) = RunCliInSubprocess([ + "suggestions", "list", "--db", fixture.DbPath, + "--query", $"api-token={rawSecret}", "--count", "--json" + ]); + var (zeroPageExitCode, zeroPageStdout, zeroPageStderr) = RunCliInSubprocess([ + "suggestions", "list", "--db", fixture.DbPath, + "--query", "bulk-marker-5061", "--compact", "--limit", "0" + ]); + + Assert.Equal(CommandExitCodes.Success, countExitCode); + Assert.Equal(string.Empty, countStderr); + using var countDoc = JsonDocument.Parse(countStdout); + Assert.Equal("count", countDoc.RootElement.GetProperty("mode").GetString()); + Assert.Equal(24, countDoc.RootElement.GetProperty("count").GetInt32()); + Assert.True(countDoc.RootElement.GetProperty("total_count_authoritative").GetBoolean()); + Assert.Equal(0, countDoc.RootElement.GetProperty("results").GetArrayLength()); + Assert.Equal(0, countDoc.RootElement.GetProperty("pagination_omitted_count").GetInt32()); + Assert.Equal(24, countDoc.RootElement.GetProperty("projection_omitted_count").GetInt32()); + Assert.True(Encoding.UTF8.GetByteCount(countStdout) < 1024); + + Assert.Equal(CommandExitCodes.Success, summaryExitCode); + Assert.Equal(string.Empty, summaryStderr); + using var summaryDoc = JsonDocument.Parse(summaryStdout); + Assert.Equal("summary", summaryDoc.RootElement.GetProperty("mode").GetString()); + Assert.Equal(24, summaryDoc.RootElement.GetProperty("total_count").GetInt32()); + var summary = summaryDoc.RootElement.GetProperty("summary"); + Assert.Equal(24, summary.GetProperty("by_status").GetProperty("counts").GetProperty("draft").GetInt32()); + Assert.Equal(12, summary.GetProperty("by_category").GetProperty("counts").GetProperty("output_format").GetInt32()); + Assert.Equal(12, summary.GetProperty("by_category").GetProperty("counts").GetProperty("language_support").GetInt32()); + Assert.Equal(0, summaryDoc.RootElement.GetProperty("results").GetArrayLength()); + Assert.Equal(0, summaryDoc.RootElement.GetProperty("pagination_omitted_count").GetInt32()); + Assert.Equal(24, summaryDoc.RootElement.GetProperty("projection_omitted_count").GetInt32()); + Assert.True(Encoding.UTF8.GetByteCount(summaryStdout) < 4096); + + Assert.Equal(CommandExitCodes.Success, compactExitCode); + Assert.Equal(string.Empty, compactStderr); + Assert.DoesNotContain(rawSecret, compactStdout, StringComparison.Ordinal); + using var compactDoc = JsonDocument.Parse(compactStdout); + var compactItem = compactDoc.RootElement.GetProperty("results")[0]; + Assert.Equal(4, compactItem.EnumerateObject().Count()); + Assert.False(compactItem.TryGetProperty("description", out _)); + Assert.False(compactItem.TryGetProperty("category", out _)); + Assert.False(compactItem.TryGetProperty("language", out _)); + Assert.Contains( + compactDoc.RootElement.GetProperty("results").EnumerateArray(), + item => item.GetProperty("title").GetString()!.Contains("redact", StringComparison.OrdinalIgnoreCase)); + + Assert.Equal(CommandExitCodes.Success, boundedExitCode); + Assert.Equal(string.Empty, boundedStderr); + Assert.True(Encoding.UTF8.GetByteCount(boundedStdout) <= byteBudget); + using var boundedDoc = JsonDocument.Parse(boundedStdout); + Assert.True(boundedDoc.RootElement.GetProperty("truncated").GetBoolean()); + Assert.True(boundedDoc.RootElement.GetProperty("byte_limit_omitted_count").GetInt32() > 0); + Assert.True(boundedDoc.RootElement.GetProperty("returned_count").GetInt32() < 24); + Assert.True(boundedDoc.RootElement.GetProperty("has_more").GetBoolean()); + Assert.True(boundedDoc.RootElement.GetProperty("next_offset").GetInt32() > 0); + Assert.Contains("--offset", boundedDoc.RootElement.GetProperty("recovery_guidance").GetString()); + + Assert.Equal(CommandExitCodes.Success, secretExitCode); + Assert.Equal(string.Empty, secretStderr); + using var secretDoc = JsonDocument.Parse(secretStdout); + Assert.Equal(0, secretDoc.RootElement.GetProperty("count").GetInt32()); + Assert.DoesNotContain(rawSecret, secretStdout, StringComparison.Ordinal); + + Assert.Equal(CommandExitCodes.UsageError, zeroPageExitCode); + Assert.Equal(string.Empty, zeroPageStderr); + Assert.Contains("--limit 0", zeroPageStdout, StringComparison.Ordinal); + } + [ProductionRuntimeFact] public void Suggestions_ListDefaultVerbAcceptsTopLevelJsonFlags_Issue4171() { diff --git a/tests/CodeIndex.Tests/golden/suggestions-compact.json b/tests/CodeIndex.Tests/golden/suggestions-compact.json new file mode 100644 index 000000000..f45e997c3 --- /dev/null +++ b/tests/CodeIndex.Tests/golden/suggestions-compact.json @@ -0,0 +1,26 @@ +{ + "api_version": "1", + "mode": "compact", + "query": "snapshot query", + "total_count": 1, + "total_count_authoritative": true, + "returned_count": 1, + "offset": 0, + "omitted_count": 0, + "pagination_omitted_count": 0, + "byte_limit_omitted_count": 0, + "projection_omitted_count": 0, + "truncated": false, + "has_more": false, + "next_offset": null, + "results": [ + { + "id": "1111111111111111111111111111111111111111111111111111111111111111", + "title": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...", + "status": "draft", + "evidence_paths": [ + "src/Snapshot.cs" + ] + } + ] +}