Skip to content

{Containerapp} Refresh validated test recordings - #10260

Merged
Julie Zhu (yanzhudd) merged 1 commit into
Azure:mainfrom
ShichaoQiu:shiqiu/refresh-containerapp-recordings
Aug 31, 2026
Merged

{Containerapp} Refresh validated test recordings#10260
Julie Zhu (yanzhudd) merged 1 commit into
Azure:mainfrom
ShichaoQiu:shiqiu/refresh-containerapp-recordings

Conversation

@ShichaoQiu

@ShichaoQiu Shichao Qiu (ShichaoQiu) commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

🤖 PR Validation — ️✔️ All clear

Breaking Changes
️✔️ None

🤖 PR Validation — ✔️ All clear

Breaking Changes
✔️ None

Summary

Refresh all 47 targeted Container Apps test recordings against the current service behavior.

This PR contains only test assets:

  • 47 refreshed recording files.
  • 4 Python test files with the minimum changes required for reliable live runs and playback.
  • No Container Apps extension product-code changes.

Python test changes

File Change Why
test_containerapp_commands.py Change the stage fallback from East US to Brazil South in seven region-sensitive cases. Add an explicit traffic entry to the Labels-mode YAML update. The refreshed recordings contain Brazil South shared-environment paths. Labels mode also requires a valid traffic allocation whose weights total 100.
test_containerapp_mount_secret_volume.py Update two assertions from None to []. The current RP response represents a secret volume that mounts all secrets with an empty secrets list. The meaning is unchanged; only the response shape changed.
test_containerappjob_with_identity.py Change one stage fallback to Brazil South. Create three role assignments with --assignee-object-id and --assignee-principal-type ServicePrincipal. Using the known principal ID avoids Microsoft Graph name/object resolution while a newly created managed identity is still replicating. This removes an eventual-consistency race without changing the assigned role or scope.
test_containerappjob_with_yaml.py Change the event-triggered YAML case fallback from East US to Brazil South. Its shared environment is location-specific and must match the recorded resource paths.

Why the region fallbacks changed

TEST_LOCATION falls back to northcentralusstage when CLITestLocation is not set, but managed identity is unavailable in that stage region. These tests therefore already had a public-region fallback.

East US could not be used for this recording refresh because managed-environment creation returned ManagedEnvironmentCapacityHeavyUsageError / AKSCapacityHeavyUsage. The same cases completed in Brazil South.

The selected location is observable during playback: prepare_containerapp_env_for_app_e2e_tests embeds it in resource group and environment names such as client.env_rg_brazilsouth/env-brazilsouth, and VCR matches those request paths. containerapp up also validates the supplied environment location. Therefore, the nine affected tests must use the same Brazil South fallback as their recordings. A replay experiment changing all ten candidate fallbacks back to East US produced nine failures; the one replay-safe case, test_containerapp_identity_registry, remains on East US.

Why Labels traffic is explicit

The YAML test updates an existing app in activeRevisionsMode: labels with targetLabel: label1. The current service requires the ingress traffic weights to total 100. The test now explicitly sends 100% of traffic to the latest revision with label1, instead of depending on an implicit service default:

traffic:
  - latestRevision: true
    weight: 100
    label: label1

Why role assignment uses object IDs

role assignment create --assignee may query Microsoft Graph to resolve the assignee. Immediately after a managed identity is created, that lookup can fail until the identity has replicated. The tests already receive each identity's principalId, so passing it through --assignee-object-id with --assignee-principal-type ServicePrincipal avoids the lookup and makes both live execution and recording deterministic.

Recording sanitization

  • Masked ACR credentials and replaced Log Analytics keys, Storage keys, and Container App auth tokens with replay-safe placeholders.
  • Removed certificate private-key material while retaining the public test certificate required by playback.
  • Reduced Graph /me and all 35 directoryObjects/getByIds responses to the synthetic fields required by the tests.
  • Verified every body-bearing Content-Length header against the UTF-8 byte length.
  • Verified all 47 recordings contain 6,084 balanced request/response interactions.
  • High- and medium-confidence azdev scan found no secrets in the final 51 files.

Validation

  • Recorded each of the 47 target cases with azdev test --live --series.
  • Replayed all 47 target cases successfully.
  • Replayed test_containerapp_create_with_yaml after the final multiline YAML cleanup: passed.
  • Compiled all four Python files.
  • azdev style containerapp, editor diagnostics, and git diff --check pass.
  • The PR is squashed to one sanitized commit: 52f6ab05d.
  • Final CI build 343233 is running; credential scan, policy, init-file, external-URL, GitHub style, and GitHub linter checks have passed.

@azure-client-tools-bot-prd

Copy link
Copy Markdown

Hi Shichao Qiu (@ShichaoQiu),
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in pyproject.toml (or setup.py, if the extension has not migrated yet) as well.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Aug 27, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Thank you for your contribution Shichao Qiu (@ShichaoQiu)! We will review the pull request and get back to you soon.

@yonzhan

Copy link
Copy Markdown
Collaborator

Containerapp

@ShichaoQiu
Shichao Qiu (ShichaoQiu) force-pushed the shiqiu/refresh-containerapp-recordings branch 7 times, most recently from 618b107 to 48829f3 Compare August 28, 2026 03:14
@ShichaoQiu
Shichao Qiu (ShichaoQiu) force-pushed the shiqiu/refresh-containerapp-recordings branch from 48829f3 to 52f6ab0 Compare August 28, 2026 03:56
@Greedygre Greedygre (Greedygre) changed the title [Containerapp] Refresh validated test recordings {Containerapp} Refresh validated test recordings Aug 28, 2026

@Greedygre Greedygre (Greedygre) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might need to check if the recording files include log shared key or not, it might be flagged after PR merge

@ShichaoQiu Shichao Qiu (ShichaoQiu) changed the title {Containerapp} Refresh validated test recordings [Containerapp] Refresh validated test recordings Aug 28, 2026
@ShichaoQiu

Copy link
Copy Markdown
Contributor Author

Thanks Greedygre (@Greedygre), I checked this specifically.

I structurally scanned all 47 changed recording files. Log Analytics shared-key fields occur only in test_containerapp_env_la_dynamic_json.yaml and test_containerapp_env_logs_e2e.yaml. Neither file contains a real key: 27 sharedKey values are null, and every non-null primarySharedKey, secondarySharedKey, or sharedKey value is one of two deterministic synthetic placeholders. Each placeholder decodes to 64 repetitions of a single byte, and the same values are reused in both recordings. There are also no Authorization: SharedKey values.

I additionally ran azdev scan at LOW, MEDIUM, and HIGH confidence across all 47 recordings, and all passed. GitHub's targeted secret scanner also reported zero secrets for the exact placeholder shapes, and the PR Credential Scan has passed. No additional sanitization is needed.

@yanzhudd Julie Zhu (yanzhudd) changed the title [Containerapp] Refresh validated test recordings {Containerapp} Refresh validated test recordings Aug 31, 2026
@yanzhudd
Julie Zhu (yanzhudd) merged commit b49471c into Azure:main Aug 31, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-observability-squad Auto-Assign Auto assign by bot ContainerApp customer-reported Issues that are reported by GitHub users external to the Azure organization.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants