You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reusable deploy workflows read github.event_name as workflow_call, silently downgrading a release deploy to dev #1805
Inside a reusable workflow, github.event_name is the caller's event, never the literal string workflow_call. GitHub documents this:
"When a reusable workflow is triggered by a caller workflow, the github context is always associated with the caller workflow."
deploy-azure.yml's prepare job branches on it:
case "$EVENT_NAME" inworkflow_dispatch|workflow_call) ENVIRONMENT="$INPUT_ENVIRONMENT" ;;*) ENVIRONMENT=dev ;;esac
The workflow_call arm is dead code. The consequence is a silent environment downgrade on one path:
deploy-all.yml triggered by the release event sets environment=prod and passes it to deploy-azure.yml. Inside the called workflow github.event_name is release, so prepare takes the *) arm and returns dev. A release-triggered multi-cloud production deploy would deploy Azure to dev, with no error and a summary that says prod.
The workflow_dispatch path survives by luck: dispatching deploy-all.yml makes the caller's event workflow_dispatch, which the first arm happens to match.
The repo already knows about this
deploy-aws-lambda.yml:110-118 documents the behaviour and handles it, including an explicit release) TARGET_ENVIRONMENT=prod ;; arm. That is the pattern to copy.
Affected
deploy-azure.yml:88-91 (prepare)
deploy-gcp.yml:75-78
deploy-aws-fargate.yml:83-86
deploy-aws-lambda.yml:110-118 is already correct and is the reference implementation.
Because deploy-all.yml fans out to all of them, a release-triggered multi-cloud deploy silently targets dev on three clouds while the summary reports prod. That is the real blast radius, and it is why this is worth fixing rather than just documenting.
Evidence status
Documentation plus the existing in-repo comment. No empirical run: gh run list --workflow deploy-all.yml returns [], so deploy-all.yml has never executed and the release path has never been exercised. That is also why this has not bitten anyone yet.
Not a concurrency bug
Found while reviewing #1801 / #1803. It does not affect that fix: the concurrency group and the state key both derive from the same needs.prepare.outputs.environment, so whatever prepare decides, the group always names the blob that actually gets written. This changes which environment is deployed, never whether two writers of one blob serialize.
Recording the reason rather than changing labels silently.
I verified the full path on origin/main and the consequence is larger than the original filing conveys: the documented production deploy mechanism silently deploys to dev and reports prod.
The chain:
deploy-all.yml:14 documents the intent: "Release creation (deploys to prod across all clouds)"
deploy-all.yml triggers on release: types: [created] as well as workflow_dispatch
on the release path, deploy-all.yml:61 sets environment=prod and passes it to each called workflow (:127, :144, :159, :174)
deploy-azure.yml's Determine environment reads EVENT_NAME: ${{ github.event_name }} and branches:
Inside a reusable workflow, github.event_name is the caller's event. On the release path it is release, so the workflow_call arm never matches, $INPUT_ENVIRONMENT (= prod) is discarded, and ENVIRONMENT becomes dev.
Note the workflow_dispatch path works by luck rather than design: the caller's event there really is workflow_dispatch, which happens to match the arm. So the bug is invisible in exactly the way that keeps it alive - the manual path anyone would test with behaves correctly.
Why this is severity/high
Creating a release does not deploy prod. Every release since this was introduced deployed dev instead.
It reports success for prod.deploy-all.yml labels the job with the environment it computed (prod), while the called workflow acted on dev. Operators reading the run see a green prod deploy that never happened.
It is silent. No error, no warning, no mismatch surfaced. The only way to notice is to compare the deployed image against the release tag.
It plausibly affects users, not just internals: fixes believed shipped to production were not. I have left impact/internal rather than raising it, because whether prod is deployed another way is a question for someone with operational context. If prod is in fact released this way, impact should go up too.
Not raising to p0: there is a working alternative path (workflow_dispatch with an explicit environment), nothing is exposed or corrupted, and no active incident is attributed to it. p1/this-sprint reflects "the production deploy path is broken and lying about it" without claiming it needs someone woken up.
Fix note
Do not simply add release to the case arm. The correct signal for "was I called by another workflow" is not github.event_name at all, since that context always describes the triggering event of the whole run. Prefer branching on whether inputs.environment is set, or make the input required on the workflow_call path so an unset value fails loudly instead of defaulting to dev. Any fix must also cover the sibling deploy workflows (deploy-aws-lambda.yml, deploy-gcp.yml) if they share the pattern.
Whatever lands, the verification is: trigger the release path and confirm the target environment is prod end to end, not merely that the job label says so. The current bug is precisely a label saying prod over an apply against dev.
PR #1803 is unaffected: its concurrency group and the state key are derived from the same value, so they stay in lockstep whatever prepare decides. This bug changes which environment is chosen, not whether the group and the blob agree.
Correction: nothing was ever mis-deployed. This bug is latent, not historical.
In my re-triage comment I wrote that "every release since this was introduced deployed dev instead" and that it "reports success for prod". Both statements are wrong, and I am retracting them.
gh run list --workflow=deploy-all.yml returns []. deploy-all.yml has never executed, on any trigger, ever. No release has gone through this path, so no deployment has ever been misrouted and no run has ever reported a prod deploy that went to dev.
The defect itself is unchanged and still real by inspection: the resolution logic in the called workflows genuinely would land on dev for a release-triggered run passing environment=prod. What is wrong is my claim that it had done so. I stated a production-impact history that does not exist.
Also corrected: the live blast radius was two clouds, not three
deploy-all.yml:82 hardcodes deploy-aws-fargate=false, so a release fans out to Lambda, GCP and Azure only. And deploy-aws-lambda.yml already resolved prod, though for the wrong reason - it hardcoded release) prod rather than honouring the input.
So the paths that would actually have misrouted are Azure and GCP. Fargate was latently broken and is fixed anyway; Lambda produced the right answer by accident.
Does this change the priority?
I am leaving it at p1, but the reasoning has to change, and anyone who disagrees should say so now that the facts are straight.
The case for p1 is no longer "this has been silently breaking production releases". It is that the documented production deploy path (deploy-all.yml:14, "Release creation (deploys to prod across all clouds)") is booby-trapped: the first time anyone uses it as intended, two clouds silently deploy dev while the run summary says prod. A latent defect on the production deploy path, which fails silently and looks green, is worth fixing before someone reaches for it.
The case for p2 is that it has zero history, zero user impact to date, and the workflow is evidently not in use.
I lean p1 on the grounds that the cost of it firing once is high and the trigger is a single click. But that is a judgement about future risk, not a report of past damage, and my earlier comment presented it as the latter.
Note on required: true
My earlier verification that environment is declared required: true on workflow_call in all the called workflows was correct but insufficient, and I want that recorded rather than left as an implied guarantee. required: true demands that the key be present, not that the value be non-empty. A caller can still pass an empty string and satisfy the schema.
So the schema does not close this. The fail-loud branch in PR #1809 does, by refusing when the resolved environment is empty rather than defaulting.
Activity
cristim commentedon Aug 12, 2026
Re-triaging p2/medium/this-quarter -> p1/high/this-sprint
Recording the reason rather than changing labels silently.
I verified the full path on
origin/mainand the consequence is larger than the original filing conveys: the documented production deploy mechanism silently deploys to dev and reports prod.The chain:
deploy-all.yml:14documents the intent: "Release creation (deploys to prod across all clouds)"deploy-all.ymltriggers onrelease: types: [created]as well asworkflow_dispatchdeploy-all.yml:61setsenvironment=prodand passes it to each called workflow (:127,:144,:159,:174)deploy-azure.yml'sDetermine environmentreadsEVENT_NAME: ${{ github.event_name }}and branches:Inside a reusable workflow,
github.event_nameis the caller's event. On the release path it isrelease, so theworkflow_callarm never matches,$INPUT_ENVIRONMENT(=prod) is discarded, andENVIRONMENTbecomesdev.Note the
workflow_dispatchpath works by luck rather than design: the caller's event there really isworkflow_dispatch, which happens to match the arm. So the bug is invisible in exactly the way that keeps it alive - the manual path anyone would test with behaves correctly.Why this is severity/high
deploy-all.ymllabels the job with the environment it computed (prod), while the called workflow acted ondev. Operators reading the run see a green prod deploy that never happened.impact/internalrather than raising it, because whether prod is deployed another way is a question for someone with operational context. If prod is in fact released this way,impactshould go up too.Not raising to p0: there is a working alternative path (
workflow_dispatchwith an explicit environment), nothing is exposed or corrupted, and no active incident is attributed to it. p1/this-sprint reflects "the production deploy path is broken and lying about it" without claiming it needs someone woken up.Fix note
Do not simply add
releaseto the case arm. The correct signal for "was I called by another workflow" is notgithub.event_nameat all, since that context always describes the triggering event of the whole run. Prefer branching on whetherinputs.environmentis set, or make the input required on theworkflow_callpath so an unset value fails loudly instead of defaulting todev. Any fix must also cover the sibling deploy workflows (deploy-aws-lambda.yml,deploy-gcp.yml) if they share the pattern.Whatever lands, the verification is: trigger the release path and confirm the target environment is
prodend to end, not merely that the job label says so. The current bug is precisely a label sayingprodover an apply againstdev.Relationship to #1801
PR #1803 is unaffected: its concurrency group and the state key are derived from the same value, so they stay in lockstep whatever
preparedecides. This bug changes which environment is chosen, not whether the group and the blob agree.fix(ci): resolve deploy environment from inputs, not github.event_name
cristim commentedon Aug 12, 2026
Correction: nothing was ever mis-deployed. This bug is latent, not historical.
In my re-triage comment I wrote that "every release since this was introduced deployed dev instead" and that it "reports success for prod". Both statements are wrong, and I am retracting them.
gh run list --workflow=deploy-all.ymlreturns[].deploy-all.ymlhas never executed, on any trigger, ever. No release has gone through this path, so no deployment has ever been misrouted and no run has ever reported a prod deploy that went to dev.The defect itself is unchanged and still real by inspection: the resolution logic in the called workflows genuinely would land on
devfor a release-triggered run passingenvironment=prod. What is wrong is my claim that it had done so. I stated a production-impact history that does not exist.Also corrected: the live blast radius was two clouds, not three
deploy-all.yml:82hardcodesdeploy-aws-fargate=false, so a release fans out to Lambda, GCP and Azure only. Anddeploy-aws-lambda.ymlalready resolvedprod, though for the wrong reason - it hardcodedrelease) prodrather than honouring the input.So the paths that would actually have misrouted are Azure and GCP. Fargate was latently broken and is fixed anyway; Lambda produced the right answer by accident.
Does this change the priority?
I am leaving it at p1, but the reasoning has to change, and anyone who disagrees should say so now that the facts are straight.
The case for p1 is no longer "this has been silently breaking production releases". It is that the documented production deploy path (
deploy-all.yml:14, "Release creation (deploys to prod across all clouds)") is booby-trapped: the first time anyone uses it as intended, two clouds silently deploy dev while the run summary says prod. A latent defect on the production deploy path, which fails silently and looks green, is worth fixing before someone reaches for it.The case for p2 is that it has zero history, zero user impact to date, and the workflow is evidently not in use.
I lean p1 on the grounds that the cost of it firing once is high and the trigger is a single click. But that is a judgement about future risk, not a report of past damage, and my earlier comment presented it as the latter.
Note on
required: trueMy earlier verification that
environmentis declaredrequired: trueonworkflow_callin all the called workflows was correct but insufficient, and I want that recorded rather than left as an implied guarantee.required: truedemands that the key be present, not that the value be non-empty. A caller can still pass an empty string and satisfy the schema.So the schema does not close this. The fail-loud branch in PR #1809 does, by refusing when the resolved environment is empty rather than defaulting.
fix(ci): resolve deploy environment from inputs, not github.event_name
fix(ci): resolve deploy environment from inputs, not github.event_name
fix(ci): resolve deploy environment from inputs, not github.event_name