Skip to content

Arm backend: Use FP64 conv in NSS quantized ref - #23259

Merged
FabulousSuperDude merged 1 commit into
pytorch:mainfrom
FabulousSuperDude:fp64_nss
Sep 29, 2026
Merged

FabulousSuperDude merged 1 commit into
pytorch:mainfrom
FabulousSuperDude:fp64_nss

Conversation

@FabulousSuperDude

@FabulousSuperDude FabulousSuperDude commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Temporarily evaluate reference convolutions in FP64 for the random-data INT test, casting each result back to FP32 to reduce sensitivity to host-dependent bias accumulation. Keep calibration, export, and qtol unchanged.

Assert that the reference graph contains all 14 expected convolutions so the FP64 override cannot silently become ineffective. Preserve the existing quantization-stage settings when replacing the stage.

Validation: NSS random-data INT test and file-specific lint pass on Ubuntu.

Temporary workaround for MLETORCH-2609.

Authored with assistance from OpenAI Codex.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

Temporarily evaluate reference convolutions in FP64 for the
random-data INT test, casting each result back to FP32 to reduce
sensitivity to host-dependent bias accumulation. Keep calibration,
export, and qtol unchanged.

Assert that the reference graph contains all 14 expected convolutions so
the FP64 override cannot silently become ineffective. Preserve the
existing quantization-stage settings when replacing the stage.

Validation: NSS random-data INT test and file-specific lint pass on
Ubuntu.

Temporary workaround for MLETORCH-2609.

Authored with assistance from OpenAI Codex.

Signed-off-by: Sangwon Ha <sangwon.ha@arm.com>
Change-Id: Ibaed52de6562b0e990a4372d6390349183a7ee51
@pytorch-bot

pytorch-bot Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23259

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Pending, 3 Unrelated Failures

As of commit 07bfe01 with merge base bfdfe10 (image):

NEW FAILURES - The following jobs have failed:

  • Cadence Build & Test / Resolve CI docker image / resolve (gh)
    ##[error]Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.
  • pull / unittest / macos / macos-job (gh)
    RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 29, 2026
@FabulousSuperDude

Copy link
Copy Markdown
Collaborator Author

@pytorchbot label 'release notes: none'

@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Sep 29, 2026
@pytorch-bot pytorch-bot Bot added the release notes: none Do not include this in the release notes label Sep 29, 2026
@FabulousSuperDude

Copy link
Copy Markdown
Collaborator Author

@pytorchbot label 'partner: arm'

@pytorch-bot pytorch-bot Bot added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Sep 29, 2026
@FabulousSuperDude
FabulousSuperDude marked this pull request as ready for review September 29, 2026 17:48
@zingo

zingo commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

We have a lot of errors in this tests here on github (some/all? before this PR) IThis might add 2 new, I think it was 7 before and 9 now. Would be nice to land a fix before pile:ing up to much errors and PRs if we need to revert.

FAILED backends/arm/test/models/test_nfru.py::test_nfru_prequantized_vgf_INT[real_data] - OSError: [Errno 30] Read-only file system: '/mnt/hf_cache/hub/models--Arm--neural-frame-rate-upscaling'
FAILED backends/arm/test/models/test_nfru.py::test_nfru_vgf_quant_a16w8[real_data] - AttributeError: 'VgfPipeline' object has no attribute 'set_quantization_calibration'
FAILED backends/arm/test/models/test_nfru.py::test_nfru_vgf_quant[real_data-qat] - AttributeError: 'VgfPipeline' object has no attribute 'set_quantization_calibration'
FAILED backends/arm/test/models/test_nss.py::test_nss_vgf_INT[real_data-ptq] - AttributeError: 'VgfPipeline' object has no attribute 'set_quantization_calibration'
FAILED backends/arm/test/models/test_nss.py::test_nss_vgf_INT[real_data-qat] - AttributeError: 'VgfPipeline' object has no attribute 'set_quantization_calibration'
FAILED backends/arm/test/models/test_nfru.py::test_nfru_prequantized_vgf_INT[random_data] - OSError: [Errno 30] Read-only file system: '/mnt/hf_cache/hub/models--Arm--neural-frame-rate-upscaling'
FAILED backends/arm/test/models/test_nfru.py::test_nfru_vgf_quant[real_data-ptq] - AttributeError: 'VgfPipeline' object has no attribute 'set_quantization_calibration'
FAILED backends/arm/test/models/test_nss.py::test_nss_prequantized_vgf_INT[real_data] - OSError: [Errno 30] Read-only file system: '/mnt/hf_cache/hub/tmp5cxy_q13'
FAILED backends/arm/test/models/test_nss.py::test_nss_prequantized_vgf_INT[random_data] - OSError: [Errno 30] Read-only file system: '/mnt/hf_cache/hub/tmpt88ue_7l'
= 9 failed, 134 passed, 5 skipped, 89 warnings, 18 rerun in 692.20s (0:11:32) ==

@FabulousSuperDude
FabulousSuperDude merged commit 06dc587 into pytorch:main Sep 29, 2026
450 of 465 checks passed
@FabulousSuperDude
FabulousSuperDude deleted the fp64_nss branch September 29, 2026 21:00
@zingo

zingo commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Did you got the Arm tests green before merging?
I see Meta seem blocked by them alos and trying to fix it also see discussions in #23182 for more info.

@zingo

zingo commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Oh I still see you had

[pull / test-arm-backend-no-driver (test_pytest_models_tosa) / linux-job](https://hud.pytorch.org/pr/pytorch/executorch/23259#109540291065) ([gh](https://github.com/pytorch/executorch/actions/runs/36607501657/job/109540291065)) ([trunk failure](https://hud.pytorch.org/pytorch/executorch/commit/bfdfe1072fa7406ba3e5e121f89d271a5b624698#109533651389))
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
[trunk / test-arm-backend-vkml (test_pytest_models_vkml) / linux-job](https://hud.pytorch.org/pr/pytorch/executorch/23259#109540352941) ([gh](https://github.com/pytorch/executorch/actions/runs/36607533120/job/109540352941)) ([trunk failure](https://hud.pytorch.org/pytorch/executorch/commit/bfdfe1072fa7406ba3e5e121f89d271a5b624698#109533628759))
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

Bummer.

@zingo

zingo commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Meta tried to fix the problem during the night and manage to fix 2/3 problems
#23265
#23268
Left after this are merged seem to only be a few numerical/high "mean abs error" problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: arm Issues related to arm backend partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: none Do not include this in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants