Skip to content

[BREAKING] FEAT: Add fixed egress deployment for CoPyRIT - #2438

Open
Bashir Partovi (bashirpartovi) wants to merge 7 commits into
microsoft:mainfrom
bashirpartovi:feature/aca-internal-nat-baseline
Open

[BREAKING] FEAT: Add fixed egress deployment for CoPyRIT#2438
Bashir Partovi (bashirpartovi) wants to merge 7 commits into
microsoft:mainfrom
bashirpartovi:feature/aca-internal-nat-baseline

Conversation

@bashirpartovi

@bashirpartovi Bashir Partovi (bashirpartovi) commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Description

This PR replaces the former Azure Container Apps environment Private Endpoint and private DNS path with a supported workload-profiles ACA topology on a dedicated delegated subnet. A Standard NAT Gateway and static public IP provide fixed outbound egress independently of the inbound routing mode.

The Bicep template supports these inbound modes:

  • Community default: direct ACA-managed HTTPS ingress, with an optional source CIDR restriction.
  • Optional Azure Front Door Premium with the public ACA hostname as its HTTPS origin.
  • Optional Azure Front Door Premium with an AFD-managed Private Link origin. When ACA public access is also disabled, Front Door becomes the only public application path.

The Front Door and Private Link controls are opt-in and default to disabled for community deployments. Bicep rejects ACA public-access shutdown unless both Front Door and its Private Link origin are enabled. Front Door changes inbound routing only; ACA-originated outbound traffic continues to use the NAT Gateway's static public IP.

The Azure DevOps deployment workflow now:

  • builds and propagates an immutable image digest;
  • validates existing application and network resources before deployment;
  • runs a fail-closed ARM what-if validator that rejects malformed results, deletes, cross-resource-group writes, protected-topology changes, and unexpected core-resource creation;
  • preserves resource and policy-managed public-IP tags;
  • optionally applies a CanNotDelete lock to the static egress public IP and verifies its resource ID and address after deployment;
  • deploys the team-managed environments with Front Door Private Link and ACA public access disabled;
  • configures the Front Door origin response timeout to 240 seconds, matching the ACA HTTP ingress limit for long-running model responses;
  • discovers and approves only the expected ACA-side private connection through a dedicated Bicep module;
  • allows up to 30 minutes for Front Door propagation, then validates ACA-side approval, Front Door health, revision health, direct-origin shutdown, and unchanged fixed egress;
  • restores the public Front Door origin, removes the private connection, and re-enables ACA public access in the required order if cutover validation fails;
  • keeps production opt-in, test-first, and independently approved.

The isolated-instance lifecycle scripts now validate Azure CLI JSON boundaries, configure SQL and Storage firewalls with the static egress IP, use ownership tags, and require explicit acknowledgement before teardown releases an allow-listed IP.

Breaking change: the existing enablePrivateEndpoint parameter and its ACA environment Private Endpoint and private DNS resources are removed, along with the previous Private Endpoint subnet parameters. ACA environment network type is creation-time configuration, so deployments using that path must migrate to a parallel VNet-integrated workload-profiles environment. Pipeline callers must adopt the deployment parameters documented in infra/README.md.

Tests and Documentation

  • python -m unittest discover -s tests/unit/infra -p 'test_*.py' -q - 24 tests pass.
  • Compiled infra/main.bicep, infra/modules/aca_nat_network.bicep, infra/modules/aca_front_door.bicep, and infra/modules/aca_private_endpoint_approval.bicep with Azure CLI/Bicep.
  • Ran py_compile for both lifecycle scripts and the what-if validator.
  • Ran bash -n for the container startup and deployment scripts.
  • Validated both parameter JSON files, README Mermaid rendering, documented Bash fences, no-wrap Markdown formatting, and git diff --check.
  • Pylance/editor diagnostics are clean for the changed lifecycle, pipeline, test, and documentation files.
  • Updated infra/README.md and infra/DEPLOY_NEW_INSTANCE.md with the supported topology, ingress and egress behavior, migration boundary, pipeline contract, lifecycle safeguards, and teardown requirements.
  • JupyText was not run because this change does not modify notebooks.

Bashir Partovi added 3 commits August 19, 2026 23:46
Deploy the GUI on a VNet-integrated public Container Apps environment with static NAT egress and optional Azure Front Door.

Harden the Azure DevOps workflow around immutable image digests, fail-closed what-if validation, and preservation of the reserved public IP. Update isolated-instance lifecycle guidance and add focused infrastructure tests.
Apply the repository-pinned Ruff formatter, resolve typing-only imports and long fixture lines, and normalize file endings for the new infrastructure Python files.
Add the trailing newlines required by the cross-platform end-of-file pre-commit hook.
Comment thread docker/start.sh
python -c "import pyrit; print(f'Running PyRIT version: {pyrit.__version__}')"

# Write .env file from PYRIT_ENV_CONTENTS (injected as the Container App's
# inline `env-file` secret; previously a Key Vault secretRef, but ACA isn't on

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this no longer apply, i.e., is ACA now on KV's trusted services list?

Comment thread gui-deploy.yml

- task: AzureCLI@2
displayName: 'Health check'
displayName: 'Preview, deploy, and verify production'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like there's essentially the same content here twice (once for test, once for prod) with just the PYRIT_SLOT different. We don't need to fix it in this PR because there's plenty going on as it is but maybe in the future.

@@ -0,0 +1,2 @@
# Copyright (c) Microsoft Corporation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well this directory is a welcome addition! 🥳


Do **not** rely solely on `/api/health` — it can pass on an old revision while
the new one is crashing. Run through this checklist:
Do **not** rely solely on `/api/health` — it can pass on an old revision while the new one is crashing. Run through this checklist:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also not new but this makes me sad and probably should be captured somewhere to fix

Comment thread infra/deploy_instance.py
"--account-name",
storage_account_name,
"--ip-address",
egress_ip,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i ran a copilot review on this and this concern was raised:
This allowlists the NAT public IP on the Storage firewall and then sets the default action to Deny but since CoPyRIT and the storage account are in the same Azure region, that IP rule won’t apply (Microsoft notes that [IP network rules have no effect on same-region requests](https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security-limitations#restrictions-for-ip-network-rules)).

The suggested fix is to enable Microsoft.Storage service endpoint on the ACA subnet and add a [virtual network rule](https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security#virtual-network-rules) for it before switching the firewall to Deny? Otherwise, I think CoPyRIT’s blob requests will be rejected with a 403.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on this

The NAT Gateway FAQ says NAT public IPs aren't used for same-region storage, so the IP rule won't match. One more thing: a service endpoint + VNet rule fixes ACA→Storage but not the browser - media.py says Azure media is served to the client via signed blob URLs, and SAS doesn't bypass network rules. Worth checking media renders end-to-end before Deny.

@behnam-o Behnam (behnam-o) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Generally, I would suggest not mixing "refactors" (cosmetic changes with little consequence e.g. updated comments, using effective*** instead of ***.properties.X, introducing env vars in inline scripts, changing reference ADO var groups, etc.) with the "actual feature" (which, here is, adding a AFD and wiring it up to the ACA) ... but I won't block on it :D

Bashir Partovi added 4 commits August 20, 2026 17:10
Add opt-in AFD Premium Private Link for ACA, disable direct ACA public access in the team deployment path, and automate connection approval, validation, and ordered rollback.

Preserve community defaults, protect the managed environment in what-if validation, and document the cutover lifecycle.
Approve ACA private endpoint connections through ARM instead of MSYS-sensitive resource-ID CLI calls, and use REST cleanup during rollback.

Allow the full observed Front Door propagation window, set explicit deployment-job timeouts, and cover the approval contract in tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants