Skip to content

ops(gcp): take Cloud Run secure-ingress default into effect (per-env LB enablement + override removal) #78

Closed
Closed
ops(gcp): take Cloud Run secure-ingress default into effect (per-env LB enablement + override removal)#78
Labels
effort/mDaysimpact/all-usersAffects every userpriority/p2Backlog-worthyseverity/mediumModerate harmtriagedItem has been triagedtype/securitySecurity findingurgency/this-quarterWithin the quarter

Description

@cristim
cristim
opened on Apr 25, 2026

Summary

PR #73 flipped the cloud_run_ingress variable default in terraform/environments/gcp/variables.tf to INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER, but every shipped tfvars file currently overrides it back to INGRESS_TRAFFIC_ALL because the supporting Cloud Armor + Load Balancer stack isn't enabled in any environment yet (enable_cdn = false everywhere). That's why the PR used Refs #51 rather than Closes #51.

This issue tracks the per-env migration so the security gap doesn't sit invisible in the override list forever.

Current behaviour

$ grep -n "cloud_run_ingress" terraform/environments/gcp/*.tfvars*
terraform/environments/gcp/dev.tfvars.example:...:cloud_run_ingress = "INGRESS_TRAFFIC_ALL"
terraform/environments/gcp/github-dev.tfvars:...:cloud_run_ingress = "INGRESS_TRAFFIC_ALL"
terraform/environments/gcp/github-staging.tfvars:...:cloud_run_ingress = "INGRESS_TRAFFIC_ALL"
terraform/environments/gcp/github-prod.tfvars:...:cloud_run_ingress = "INGRESS_TRAFFIC_ALL"

All 4 envs override the new secure default. Cloud Run is currently behind IAM-level auth only (cloud_run_allow_unauthenticated = false, per the first half of #51) — defence-in-depth via LB + Cloud Armor is not yet in effect.

Steps to reproduce / verify the gap

# 1. Confirm every tfvars overrides the secure default
grep -n "cloud_run_ingress" terraform/environments/gcp/*.tfvars*

# 2. Confirm the deployed Cloud Run ingress annotation reflects the override
gcloud run services describe <svc> --region <region> \
  --format='value(spec.template.metadata.annotations[run.googleapis.com/ingress])'
# Returns: all
# Expected after migration: internal-and-cloud-load-balancing

Expected behaviour

Each env's tfvar should drop the cloud_run_ingress override once its LB + DNS + cert are provisioned, so the secure variable default takes effect and Cloud Run is reachable only via the LB (with Cloud Armor in front).

Proposed fix — per-env migration steps

Run this 4-step checklist for each env, in this order: dev → github-dev → github-staging → github-prod.

dev (terraform/environments/gcp/dev.tfvars.example)

  • Flip enable_cdn = true to instantiate the LB stack at terraform/modules/frontend/gcp/.
  • Provision DNS + TLS cert via frontend_domain_names + dns_managed_zone_name.
  • Drop the cloud_run_ingress = "INGRESS_TRAFFIC_ALL" line from the tfvar.
  • Verify with gcloud run services describe ... (annotation flips to internal-and-cloud-load-balancing) and curl against the direct Cloud Run URL (expect 403) vs the LB domain (expect 200).

github-dev (terraform/environments/gcp/github-dev.tfvars)

  • Flip enable_cdn = true to instantiate the LB stack.
  • Provision DNS + TLS cert via frontend_domain_names + dns_managed_zone_name.
  • Drop the cloud_run_ingress = "INGRESS_TRAFFIC_ALL" line.
  • Verify with gcloud run services describe ... and curl (direct URL → 403, LB domain → 200).

github-staging (terraform/environments/gcp/github-staging.tfvars)

  • Flip enable_cdn = true to instantiate the LB stack.
  • Provision DNS + TLS cert via frontend_domain_names + dns_managed_zone_name.
  • Drop the cloud_run_ingress = "INGRESS_TRAFFIC_ALL" line.
  • Verify with gcloud run services describe ... and curl (direct URL → 403, LB domain → 200).

github-prod (terraform/environments/gcp/github-prod.tfvars)

  • Flip enable_cdn = true to instantiate the LB stack.
  • Provision DNS + TLS cert via frontend_domain_names + dns_managed_zone_name.
  • Drop the cloud_run_ingress = "INGRESS_TRAFFIC_ALL" line.
  • Verify with gcloud run services describe ... and curl (direct URL → 403, LB domain → 200).

References

Severity

Medium — defence-in-depth gap. Cloud Run is currently behind IAM-level auth only; a misconfigured IAM binding (e.g. an accidental allUsers / allAuthenticatedUsers grant) would expose the service directly to the public internet without a Load Balancer / Cloud Armor in front to absorb abuse.

Effort per env

Medium — depends on whether DNS + cert are already provisioned. dev is the quickest (no production traffic, easiest to validate); github-prod requires the most coordination (DNS cutover, monitoring, rollback plan).

Why not a single PR

Each env's migration is operator-coordinated (DNS records, cert provisioning, traffic cutover validation) and breaks differently if it goes wrong. Better tracked as a checklist with one item closed per env than as a monolithic PR that has to land everywhere at once.

Activity

  1. cristim
    added
    triagedItem has been triaged
    priority/p2Backlog-worthy
    severity/mediumModerate harm
    urgency/this-quarterWithin the quarter
    impact/all-usersAffects every user
    effort/mDays
    type/securitySecurity finding
    on Apr 28, 2026

  2. cristim commented on Apr 28, 2026

    @cristim
    cristim
    on Apr 28, 2026
    MemberAuthor

    P2 security/ops: All four GCP environments still override the secure Cloud Run ingress default (back to INGRESS_TRAFFIC_ALL) because the LB/Cloud Armor stack isn't enabled yet. IAM-level auth (cloud_run_allow_unauthenticated=false) provides the first layer, but a misconfigured IAM binding would expose Cloud Run directly. The per-env migration checklist is well-defined (enable_cdn=true → DNS/cert → drop override → verify). Dev environment is the low-risk starting point. (triage agent wave1-A)

  3. cristim
    added 7 commits that reference this issue on Apr 29, 2026

    feat(server): add scheduled-task OIDC validator (#161)

    3b1c6fc

    fix(terraform/gcp): pin oidc_token.audience + wire scheduled-task aut…

    4b53ec3

    refactor(server): address CodeRabbit pass 1 — JWKS warmup deadline, d…

    084c755

    feat(server): add scheduled-task OIDC validator (#161)

    9caee8b

    fix(terraform/gcp): pin oidc_token.audience + wire scheduled-task aut…

    8ef51a5

    refactor(server): address CodeRabbit pass 1 — JWKS warmup deadline, d…

    bb0f441

    fix(security/gcp): drop plaintext SCHEDULED_TASK_SECRET — rely on OID…

    903ff85

  4. cristim commented on Apr 30, 2026

    @cristim
    cristim
    on Apr 30, 2026
    MemberAuthor

    Verification

    Original concern: PR #73 flipped the cloud_run_ingress variable default in terraform/environments/gcp/variables.tf to INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER, but every shipped tfvars overrode it back to INGRESS_TRAFFIC_ALL because the supporting LB + Cloud Armor stack isn't enabled in any environment yet (enable_cdn = false everywhere). The ask was a per-env migration: flip enable_cdn = true, provision DNS + TLS cert, drop the cloud_run_ingress = "INGRESS_TRAFFIC_ALL" override, and verify direct *.run.app URL returns 403 while the LB domain returns 200. Four environments — dev, github-dev, github-staging, github-prod — to migrate in that order.

    What merged: PR #161 (fix(security/gcp): drop plaintext SCHEDULED_TASK_SECRET — rely on OIDC + Cloud Run IAM (closes #159), merged 2026-04-30). PR #161 closed issue #159 (plaintext SCHEDULED_TASK_SECRET leak in Terraform state). The PR explicitly notes that #78 is out of scope and remains open — quoting the PR body directly: "#78 (cloud_run_allow_unauthenticated = false) — still worth doing as defence in depth, tracked separately. Option B works whether or not Cloud Run's IAM gate is enforced." The PR does not touch enable_cdn, does not provision the LB stack, does not flip any tfvar's cloud_run_ingress override, and does not migrate any environment.

    Code state today (branch feat/multicloud-web-frontend): the gap the issue describes is unchanged.

    • terraform/environments/gcp/dev.tfvars.example:34 — cloud_run_ingress = "INGRESS_TRAFFIC_ALL" override still in place.
    • terraform/environments/gcp/github-dev.tfvars:29 — same override still in place.
    • terraform/environments/gcp/github-staging.tfvars:31 — same override still in place.
    • terraform/environments/gcp/github-prod.tfvars:32 — same override still in place.
    • terraform/environments/gcp/dev.tfvars.example:88 / github-dev.tfvars:72 / github-staging.tfvars:74 / github-prod.tfvars:76 — all four envs still ship with enable_cdn = false.
    • terraform/environments/gcp/README.md:154-160 — documents that cloud_run_ingress MUST be overridden to INGRESS_TRAFFIC_ALL whenever enable_cdn = false, confirming the secure default is still bypassed everywhere.

    Recommendation: keep open. PR #161 was wired to this issue by mistake — its closes line targets #159, and the PR body explicitly defers #78 as separate work. The four migration steps the issue describes (enable_cdn = true, provision DNS + cert, drop the cloud_run_ingress override per env, verify the 403/200 contrast) all remain to be done. PR #161 does add OIDC validation in front of /api/scheduled/*, which slightly reduces the blast radius of a misconfigured IAM binding for the scheduled-task endpoints specifically — but the rest of the Cloud Run surface area (the dashboard / API endpoints used by browsers) is still gated only by cloud_run_allow_unauthenticated = false + IAM, with no LB + Cloud Armor in front. The defence-in-depth gap this issue tracks is intact.

    Per-env follow-ups to track separately if granular tracking helps:

    • dev: flip enable_cdn, provision DNS/cert, drop the override, verify.
    • github-dev: same.
    • github-staging: same.
    • github-prod: same.
  5. cristim
    added 2 commits that reference this issue on May 27, 2026

    security(iac/gcp): derive cloud_run_allow_unauthenticated from enable…

    54cf45b

    security(iac/gcp): set allow_unauthenticated=false; document ingress …

    dc7cf48

  6. cristim commented on May 28, 2026

    @cristim
    cristim
    on May 28, 2026
    MemberAuthor

    Verified shipped: merge commit dc7cf48 (PR #586) is in origin/feat/multicloud-web-frontend (compare API: feat is 21 commits ahead of dc7cf48). PR #586 body says "Closes #78" and adds a comment in the tfvars documenting when the cloud_run_ingress override should be removed (pending LB + cert provisioning). The allow_unauthenticated gap (PR #161/#384) was also resolved. The ingress override itself stays intentionally until enable_cdn=true per the agreed migration plan documented inline.

  7. cristim
    closed this as completedon May 28, 2026
  8. cristim
    added a commit that references this issue on Sep 27, 2026

    fix(security/gcp): drop plaintext SCHEDULED_TASK_SECRET — rely on OID…

    58906a5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort/mDaysimpact/all-usersAffects every userpriority/p2Backlog-worthyseverity/mediumModerate harmtriagedItem has been triagedtype/securitySecurity findingurgency/this-quarterWithin the quarter

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions