Skip to content

Add Pixel 10a mobile performance coverage - #5284

Draft
LoopedBard3 wants to merge 6 commits into
dotnet:mainfrom
LoopedBard3:loopedbard3-add-pixel-10a-tests
Draft

Add Pixel 10a mobile performance coverage#5284
LoopedBard3 wants to merge 6 commits into
dotnet:mainfrom
LoopedBard3:loopedbard3-add-pixel-10a-tests

Conversation

@LoopedBard3

@LoopedBard3 LoopedBard3 commented Aug 19, 2026

Copy link
Copy Markdown
Member

Summary

  • add Pixel 10a to all existing Pixel SDK/MAUI performance test matrices
  • run the runtime Android JIT, static-linking, and R2R variants on Pixel 10a
  • map the new queue to the Pixel10a device name while preserving Pixel 8 coverage

Testing

  • parsed the changed YAML files
  • validated Python syntax and Pixel 8/Pixel 10a matrix parity

Internal Run: https://dev.azure.com/dnceng/internal/_build/results?buildId=3052379&view=results

Copilot AI lite review requested due to automatic review settings August 19, 2026 22:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the repo’s mobile performance infrastructure to include Pixel 10a alongside existing Pixel coverage by introducing a new Helix queue mapping and wiring a machinePool value through pipeline/job matrices.

Changes:

  • Added a new logical-machine → Helix queue mapping for perfpixel10a and treated the new Pixel 10a queue as a mobile queue for device-name env var handling.
  • Updated Helix/pipeline templates and matrices to pass machinePool (e.g., Pixel8, Pixel10a) and schedule Pixel 10a runs across existing SDK/MAUI matrices.
  • Refactored runtime Android perf jobs to iterate over a configurable androidMachines list (Pixel + Pixel 10a).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
scripts/run_performance_job.py Adds Pixel 10a queue mapping and includes the queue in mobile DEVICE_NAME handling.
helix.yml Adds machinePool variable and a Pixel 10a profile that passes --machine-pool.
eng/pipelines/templates/run-performance-job.yml Updates the machinePool parameter comment to include Pixel 10a.
eng/pipelines/templates/build-machine-matrix.yml Adds a private-build machine matrix entry for Pixel 10a with machinePool: Pixel10a.
eng/pipelines/sdk-perf-jobs.yml Adds the Pixel 10a machine to multiple existing SDK scenario matrices.
eng/pipelines/runtime-perf-jobs.yml Introduces androidMachines and loops Android runtime perf jobs over Pixel + Pixel 10a.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eng/pipelines/runtime-perf-jobs.yml Outdated
Copilot AI review requested due to automatic review settings August 19, 2026 22:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/pipelines/templates/build-machine-matrix.yml:137

  • The comment says "Windows ARM64 Pixel 10a", but this matrix entry is using archType: x64 and vmImage: windows-2022 (Windows x64 host) and targets an Android ARM64 device. This can mislead future edits/debugging.
- ${{ if and(containsValue(parameters.buildMachines, 'win-x64-android-arm64-pixel10a'), not(eq(parameters.isPublic, true))) }}: # Windows ARM64 Pixel 10a only used in private builds currently

scripts/run_performance_job.py:1151

  • The Pixel 10a DEVICE_NAME special-case is unnecessarily tied to both logical_machine and the queue string. If a Pixel 10a run is configured via --queue Windows.11.Amd64.Pixel.10.Perf (without --logical-machine/--machine-pool), DEVICE_NAME will not be set even though this queue is treated as a mobile queue below.
    # Pixel 10a runtime jobs use a logical machine rather than a machine pool.
    if args.logical_machine == "perfpixel10a" and args.queue == "Windows.11.Amd64.Pixel.10.Perf":
        args.run_env_vars["DEVICE_NAME"] = "Pixel10a"
    elif args.machine_pool and args.queue and args.queue in (
        "Windows.11.Amd64.Pixel.Perf",

Copilot AI review requested due to automatic review settings August 19, 2026 22:34
@LoopedBard3 LoopedBard3 self-assigned this Aug 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

@LoopedBard3

Copy link
Copy Markdown
Member Author

While this PR is good, there are some on machine changes that need to be made to get these devices fully working, so waiting on those changes before merging.

DrewScoggins
DrewScoggins previously approved these changes Aug 20, 2026
matouskozak
matouskozak previously approved these changes Aug 24, 2026

@matouskozak matouskozak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to add the devices also to the MAUI SDK scenario runs or is this a staged deployment and testing it on runtime runs first?

@LoopedBard3

Copy link
Copy Markdown
Member Author

Would it make sense to add the devices also to the MAUI SDK scenario runs or is this a staged deployment and testing it on runtime runs first?

Are you talking about in runtime or the dotnet-performance runs? This should be added to both, but I may have missed somewhere.

@matouskozak

Copy link
Copy Markdown
Member

Are you talking about in runtime or the dotnet-performance runs? This should be added to both, but I may have missed somewhere.

I meant dotnet-performance but discard my comment, I read the PR wrong (I saw the bigger changes for runtime runs and was looking for the same in Maui SDK but didn't realize the one-line change is enough). Sorry

@LoopedBard3

Copy link
Copy Markdown
Member Author

I meant dotnet-performance but discard my comment, I read the PR wrong (I saw the bigger changes for runtime runs and was looking for the same in Maui SDK but didn't realize the one-line change is enough). Sorry

All good, just confirming that the devices are properly setup to upload results then will merge 👍.

Parker Bibus and others added 4 commits August 24, 2026 14:54
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 24, 2026 21:55
@LoopedBard3
LoopedBard3 dismissed stale reviews from matouskozak and DrewScoggins via f916d88 August 24, 2026 21:55
@LoopedBard3
LoopedBard3 force-pushed the loopedbard3-add-pixel-10a-tests branch from e6b60f2 to f916d88 Compare August 24, 2026 21:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Suppressed comments (4)

eng/pipelines/runtime-perf-jobs.yml:127

  • This runtime job passes only logicalMachine, so machinePool remains empty in run-performance-job.yml and run_performance_job.py does not emit DEVICE_NAME for the new mobile queue. The job will select the Pixel 10a queue without identifying the Pixel10a device in machine setup/results. Pass machinePool: Pixel10a here (and in the other two Pixel 10a runtime blocks).
            logicalMachine: 'perfpixel10a'

eng/pipelines/runtime-perf-jobs.yml:167

  • This runtime job passes only logicalMachine, so machinePool remains empty in run-performance-job.yml and run_performance_job.py does not emit DEVICE_NAME for the new mobile queue. The job will select the Pixel 10a queue without identifying the Pixel10a device in machine setup/results. Pass machinePool: Pixel10a here (and in the other two Pixel 10a runtime blocks).
            logicalMachine: 'perfpixel10a'

eng/pipelines/runtime-perf-jobs.yml:207

  • This runtime job passes only logicalMachine, so machinePool remains empty in run-performance-job.yml and run_performance_job.py does not emit DEVICE_NAME for the new mobile queue. The job will select the Pixel 10a queue without identifying the Pixel10a device in machine setup/results. Pass machinePool: Pixel10a here (and in the other two Pixel 10a runtime blocks).
            logicalMachine: 'perfpixel10a'

src/scenarios/shared/runner.py:470

  • If the new settings get command fails, this exception is raised before setup_device initializes stopappcommand; the finally block then calls close_device(), whose unconditional RunCommand(self.stopappcommand, ...) raises ValueError and masks the original adb failure. Move this call after setup_device (it only needs to run before the procstats clear), or make teardown tolerate an incomplete setup.
                androidHelper.enable_pss_profiling()

Comment thread src/scenarios/shared/runner.py Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 24, 2026 22:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 309b3a9e-9724-411f-8460-751ec0faa6a3
Copilot AI review requested due to automatic review settings August 24, 2026 23:15
@LoopedBard3
LoopedBard3 marked this pull request as draft August 24, 2026 23:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants