Repository navigation
docs(fleet): review-thread stewardship — reply and resolve as fleet actions #611
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
51ddd82
docs(fleet): review-thread stewardship — reply and resolve as fleet a…
hyperpolymath e1a9a32
docs(fleet): clarify review-thread matching, resolution prerequisites…
coderabbitai[bot] 16a2c41
chore(fleet): bump gsbot rustls to 0.23.46 and allow Deno as a second…
coderabbitai[bot] 71c1af9
docs(robot-repo-automaton): limit Deno to runtime use and require Bun…
coderabbitai[bot] eef490c
test(fleet): add documentation policy checks for review-thread stewar…
coderabbitai[bot] 3736030
test(runtime-policy): Require Deno to use Bun for JavaScript dependen…
coderabbitai[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| #!/usr/bin/env bash | ||
| # SPDX-License-Identifier: MPL-2.0 | ||
| # Acceptance checks for the documented contract, not runtime enforcement. | ||
| # Offline: reads documentation only; never calls GitHub or enables the fleet. | ||
| # Run with: bash tests/review-thread-stewardship-test.sh | ||
| set -euo pipefail | ||
| root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
|
|
||
| # Limit assertions to the relevant section so another bot or metarule cannot | ||
| # accidentally satisfy them. Collapse wrapping without snapshotting whole docs. | ||
| section() { | ||
| local file="$1" heading="$2" level="$3" | ||
| awk -v heading="$heading" -v level="$level" ' | ||
| $0 == heading { found = 1; next } | ||
| found && $0 ~ "^" level " " { exit } | ||
| found { print } | ||
| ' "$file" | ||
| } | ||
| normalize() { awk '{$1=$1; printf "%s ", $0}'; } | ||
| require() { | ||
| local label="$1" pattern="$2" content="$3" | ||
| if ! grep -Eq -- "$pattern" <<< "$content"; then | ||
| printf 'FAIL: %s\n' "$label" >&2 | ||
| exit 1 | ||
| fi | ||
| printf 'PASS: %s\n' "$label" | ||
| } | ||
|
|
||
| operations="$(section "$root/docs/BOT-OPERATIONS.adoc" \ | ||
| '==== Review-Thread Stewardship (metarule)' '====')" | ||
| tokens="$(section "$root/docs/TOKEN-ARCHITECTURE.adoc" \ | ||
| '=== 3. echidnabot' '===')" | ||
|
|
||
| # Keep each guard in its own numbered step; a disclaimer elsewhere must not | ||
| # conceal a weakened matching, resolution, or invalid-finding instruction. | ||
| step() { | ||
| local number="$1" | ||
| awk -v number="$number" ' | ||
| /^[0-9]+\. / { | ||
| if (found) exit | ||
| if ($1 == number ".") found = 1 | ||
| } | ||
| found && /^\.\.\.\.$/ { exit } | ||
| found { print } | ||
| ' <<< "$operations" | normalize | ||
| } | ||
| match_step="$(step 1)" | ||
| fix_step="$(step 2)" | ||
| reply_step="$(step 3)" | ||
| resolve_step="$(step 4)" | ||
| invalid_step="$(step 5)" | ||
|
|
||
| require 'candidate filtering uses the same PR, file/line class and unresolved threads' \ | ||
| 'same PR and file/line class only to filter candidate unresolved' "$match_step" | ||
| require 'candidate filtering requires a verified finding-to-thread match' \ | ||
| 'Verify the finding-to-thread match' "$match_step" | ||
| require 'reply and resolution require exactly one match' \ | ||
| 'exactly one match before replying or resolving' "$match_step" | ||
| require 'zero or ambiguous matches leave every thread unchanged' \ | ||
| 'Leave all threads unchanged if zero or multiple matches' "$match_step" | ||
| require 'the executor lands the fix on the PR branch' \ | ||
| 'fix lands on the PR branch \(robot-repo-automaton' "$fix_step" | ||
| require 'identity findings cannot be automatically fixed' \ | ||
| 'HYP-S010 are auto_fixable=false' "$fix_step" | ||
| require 'identity direction belongs to the owner' \ | ||
| 'bot proposes and replies, the owner rules on direction' "$fix_step" | ||
| require 'echidnabot replies with both the fix commit and reasoning' \ | ||
| 'echidnabot replies to the thread with the fix commit and reasoning' "$reply_step" | ||
|
|
||
| # REST takes a top-level comment ID; GraphQL takes a review-thread node ID. | ||
| reply_endpoint='POST /repos/{owner}/{repo}/pulls/{pr}/comments/{id}/replies' | ||
| require 'operations document the REST reply endpoint' \ | ||
| "$(sed 's/[][\\.^$*+?(){}|]/\\&/g' <<< "$reply_endpoint")" "$reply_step" | ||
| require 'REST replies use the top-level review-comment ID' \ | ||
| 'where \{id\} is the top-level review-comment ID' "$reply_step" | ||
| require 'resolution follows both a landed fix and a posted reply' \ | ||
| "only after the required fix lands and Step 3's reply is posted" "$resolve_step" | ||
| require 'resolution uses a GraphQL thread ID from the PR' \ | ||
| "GraphQL mutation resolveReviewThread \(thread id from the PR's reviewThreads\)" "$resolve_step" | ||
| require 'identity threads wait for owner direction, approval and a landed fix' \ | ||
| 'HYP-S010 stay open until the owner directs and approves a fix and that fix lands' "$resolve_step" | ||
| require 'invalid findings take the skip path instead of normal resolution' \ | ||
| 'For verified-invalid findings, follow Step 5 instead' "$resolve_step" | ||
| require 'unaddressed findings must never be resolved' \ | ||
| 'Never resolve a thread whose finding is not addressed' "$invalid_step" | ||
| require 'invalid findings receive a skip explanation and remain unresolved' \ | ||
| 'verified as no-longer-valid, post a "skipped with a brief reason" reply explaining why and leave the thread unresolved' "$invalid_step" | ||
| require 'alignment findings require checking for template residue first' \ | ||
| 'check which side is the residue before automating' "$(normalize <<< "$operations")" | ||
|
|
||
| reply_row="$(awk '/^\| Reply to a review thread \|/ {print}' <<< "$tokens")" | ||
| resolve_row="$(awk '/^\| Resolve a review thread \|/ {print}' <<< "$tokens")" | ||
| require 'echidnabot owns exactly one reply action and one resolve action' \ | ||
| '^1 1$' "$(grep -c '^| Reply to a review thread |' <<< "$tokens") $(grep -c '^| Resolve a review thread |' <<< "$tokens")" | ||
| require 'token docs use the same REST endpoint as operations' \ | ||
| "$(sed 's/[][\\.^$*+?(){}|]/\\&/g' <<< "${reply_endpoint//\{id\}/\{comment_id\}}")" "$reply_row" | ||
| require 'token docs distinguish REST comment IDs from GraphQL thread IDs' \ | ||
| 'top-level review-comment ID, distinct from the GraphQL review-thread ID' "$reply_row" | ||
| require 'review replies require no additional token scope' \ | ||
| 'no new scope, `repo` covers it' "$reply_row" | ||
| require 'token docs name the GraphQL mutation and threadId argument' \ | ||
| 'mutation resolveReviewThread\(input: \{threadId\}\)' "$resolve_row" | ||
| require 'token docs restrict resolution to addressed findings after the reply' \ | ||
| 'resolves only threads whose finding the fleet addressed, with the fix-commit reply posted first' "$resolve_row" | ||
| require 'operations link to the token permission document' \ | ||
| 'link:TOKEN-ARCHITECTURE\.adoc\[' "$(normalize <<< "$operations")" | ||
| require 'token reply action refers back to the operations contract' \ | ||
| 'BOT-OPERATIONS\.adoc' "$reply_row" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| #!/usr/bin/env bash | ||
| # SPDX-License-Identifier: MPL-2.0 | ||
| # Acceptance check for RRA's documented runtime policy. No runtime is started. | ||
| # Run with: bash tests/runtime-policy-test.sh | ||
| set -euo pipefail | ||
| root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| policy="$root/robot-repo-automaton/.claude/CLAUDE.md" | ||
|
|
||
| section() { | ||
| local heading="$1" | ||
| awk -v heading="$heading" ' | ||
| $0 == heading { found = 1; next } | ||
| found && /^### / { exit } | ||
| found { print } | ||
| ' "$policy" | ||
| } | ||
| allowed="$(section '### ALLOWED Languages & Tools')" | ||
| banned="$(section '### BANNED - Do Not Use')" | ||
|
|
||
| if [[ -z "$allowed" || -z "$banned" ]]; then | ||
| echo 'FAIL: runtime policy must retain its allowed and banned sections' >&2 | ||
| exit 1 | ||
| fi | ||
| if [[ "$(grep -cE '^\| \*\*Deno\*\* \|' <<< "$allowed")" != 1 ]] || | ||
| ! grep -Eq '^\| \*\*Deno\*\* \| JS runtime only \(tier 2\) \| Secondary to Bun; limited to running compiled ESM/JS\.' <<< "$allowed"; then | ||
| echo 'FAIL: Deno must be allowed once as the secondary tier 2 runtime' >&2 | ||
| exit 1 | ||
| fi | ||
| echo 'PASS: Deno is allowed as the secondary tier 2 runtime' | ||
| if ! grep -Eq '^\| \*\*Deno\*\* \|[^|]*\|[^|]*Use Bun for all JavaScript dependency installation and management\.' <<< "$allowed"; then | ||
| echo 'FAIL: Deno must delegate JavaScript dependency installation and management to Bun' >&2 | ||
| exit 1 | ||
| fi | ||
| echo 'PASS: JavaScript dependency installation and management remain with Bun' | ||
| if ! grep -Eq '^\| \*\*Bun\*\* \| JS runtime & package management \(tier 1\) \|' <<< "$allowed"; then | ||
| echo 'FAIL: Bun must remain the primary tier 1 runtime' >&2 | ||
| exit 1 | ||
| fi | ||
| echo 'PASS: Bun remains the primary tier 1 runtime' | ||
| # Regression: adding Deno to ALLOWED alone leaves contradictory instructions. | ||
| if grep -Eiq '^\|[[:space:]*`]*Deno[[:space:]*`]*\|' <<< "$banned"; then | ||
| echo 'FAIL: Deno must not also appear in the banned runtime table' >&2 | ||
| exit 1 | ||
| fi | ||
| echo 'PASS: the banned table does not contradict the Deno allowance' |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.