Skip to content

feat: PR validation includes release worthiness notification (W-23251216) - #177

Merged
WillieRuemmele merged 6 commits into
mainfrom
d/W-23251216-b
Aug 28, 2026
Merged

feat: PR validation includes release worthiness notification (W-23251216)#177
WillieRuemmele merged 6 commits into
mainfrom
d/W-23251216-b

Conversation

@jfeingold35

@jfeingold35 jfeingold35 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Previously, the validatePR.yml workflow only checked that a Pull Request's title and body both contained a string that looks liked it could conceivably be a GUS Work Item (i.e., W-\d{7,8} in title, and @W-\d{7,8}@ in body).

Now, the workflow enforces multiple things.

  1. The Title contains a string that looks like it could be a work item.
  2. The Body contains the same string from the Title, wrapped in @'s.
  3. If those criteria are violated, a criteria tree is logged with checks and x's to show what you did wrong.
  4. An additional Job has been added that, if none of the commits in the PR are feat: ___ or fix: ___, adds a comment to the PR saying, more or less, "this PR will not trigger a release, use 'fix' or 'feat' to do that; to silence additional messages like this, react with the 👀 emoji."

Criteria 1-3 are derived from #162 , a first attempt at this feature.
Criteria 4 was experimentally written and tested in #176 and #178 and salesforcecli/testPackageRelease#63.

This enhancement was done for @W-23251216@.

@WillieRuemmele

Copy link
Copy Markdown
Contributor

Bug: typo on line 53 — steps.regex.match-gus-wi-body.outputs.match uses a . instead of - between regex and match. This means BODY_WI will always evaluate to the FAIL icon (empty string → $FAIL) regardless of whether the body actually contains the work item.

Should be:

-          [[ '${{ steps.regex.match-gus-wi-body.outputs.match }}' == '' ]] && BODY_WI=$FAIL || BODY_WI=$PASS
+          [[ '${{ steps.regex-match-gus-wi-body.outputs.match }}' == '' ]] && BODY_WI=$FAIL || BODY_WI=$PASS

@WillieRuemmele

Copy link
Copy Markdown
Contributor

QA Results from plugin-user#1510

Pointed validate-pr.yml at this branch to test. Results from run 33205084402:

Job 1: pr-validation (criteria tree) ✓ works

Title and body both missing WI → both show ✕ as expected:

- [✕] PR Title must include include a Work Item number
- [✕] PR Body must include Work Item number from Title, wrapped in '@'s

Note: "include include" (typo — doubled word in the message).

Job 2: notify-of-release-worthiness ✕ crashes

SyntaxError: Invalid or unexpected token
    at new AsyncFunction (<anonymous>)
    at callAsyncFunction (.../actions/github-script/v7/dist/index.js:36187:16)

The roboCommentBody string contains characters that break when the YAML-embedded JS is compiled by actions/github-script. The escaped \" sequences in the string may be getting double-interpreted. Might need to define the message body differently (e.g., as a separate env: variable, or use template literals with proper escaping).

Summary of bugs found

  1. Line 53 typo: steps.regex.match-gus-wi-body (. instead of -) — BODY_WI always shows ✕ (previous comment)
  2. "include include": doubled word in criteria message
  3. notify-of-release-worthiness SyntaxError: inline script fails to compile at runtime

@WillieRuemmele

Copy link
Copy Markdown
Contributor

QA Results — Round 2 (after fixes)

Tested via plugin-user#1510.

What's fixed

  • notify-of-release-worthiness SyntaxError — resolved, job runs successfully now

Test scenarios

# Scenario Expected Actual
1 No WI in title or body Fail, both ✕ ✓ Pass
2 Matching WI in title + body (W-24022936 / @W-24022936@) Pass ✓ Pass
3 WI in title, different WI in body Fail, title ✔ body ✕ ✓ Pass
4 [skip-validate-pr] in title, no WI Pass (skipped) ✓ Pass
5 chore: only commits, no prior 👀 Bot comments "won't trigger release" ✓ Pass
6 Re-trigger after 👀 reaction on bot comment No new comment ✓ Pass

Remaining issues

  1. Line 53 typosteps.regex.match-gus-wi-body (. vs -). The body ✕/✔ indicator in the criteria tree is unreliable because of this. In scenario 3, the body showed ✕ but we can't tell if that's the typo or a real detection.
  2. "include include" — doubled word in the criteria message.
  3. Duplicate bot comments — the release-worthiness job posted the comment 4 times across 4 workflow runs (once per trigger). It only suppresses if the existing comment has an 👀 reaction. Consider also checking if any prior bot comment with the same body already exists (regardless of reactions) to avoid spamming.

- Fix "include include" doubled word in the PR validation warning
- Prevent the release-worthiness bot from posting duplicate comments
  on every workflow re-trigger; now checks if an identical comment
  already exists before posting a new one
@WillieRuemmele

Copy link
Copy Markdown
Contributor

QA Results — Final (all fixes verified)

Tested via plugin-user#1510 pointing at this branch.

All scenarios passing

# Scenario Result
1 No WI in title or body ✓ Fails with correct criteria tree (both ✕)
2 Matching WI in title + body ✓ Passes
3 WI in title, different WI in body ✓ Fails (title ✔, body ✕)
4 [skip-validate-pr] in title ✓ Passes (skipped)
5 chore: only commits → bot comment ✓ Posts release-worthiness warning
6 Re-trigger after 👀 reaction ✓ No new comment
7 Re-trigger without 👀 (duplicate suppression) ✓ No duplicate comment posted

Issues found and resolved

Issue Fix
steps.regex.match-gus-wi-body typo (. vs -) Fixed by Jamie
SyntaxError in release-worthiness inline JS Fixed by Jamie
"include include" doubled word Fixed in 6e6d7d6
Duplicate bot comments on every re-trigger Fixed in 6e6d7d6

All clear — approving.

@WillieRuemmele WillieRuemmele left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA verified — all scenarios passing, fixes confirmed.

@WillieRuemmele
WillieRuemmele merged commit 9adaf89 into main Aug 28, 2026
3 checks passed
@WillieRuemmele
WillieRuemmele deleted the d/W-23251216-b branch August 28, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants