Skip to content

performance(ring para): change sdpa backend to flash attention - #267

Open
JasonHe-WQ wants to merge 1 commit into
modelscope:v1from
JasonHe-WQ:dev/cp_flash
Open

performance(ring para): change sdpa backend to flash attention#267
JasonHe-WQ wants to merge 1 commit into
modelscope:v1from
JasonHe-WQ:dev/cp_flash

Conversation

@JasonHe-WQ

Copy link
Copy Markdown
Collaborator

Description

When using the sdpa attention backend with ring sp, this pr replaces efficient backend with flash backend to accelerate.

Related Issue

#266

Test Env

config val
Python / PyTorch 3.12.13 / 2.11.0+cu130
CUDA / NCCL 13.0 / 2.28.9
Diffusers / Transformers 0.36.0 / 4.57.6
GPU 4 x NVIDIA RTX PRO 5000 72GB Blackwell,driver 580.159.03
Attention backend DiffSynth sdpa
Dtype BF16
Picture 1024 x 1024,1 per prompt
Denoise 5 steps,CFG scale 4.0,seed 42
Prompt A red fox sitting beside a mountain lake at sunrise, highly detailed
Negative prompt blurry, low quality

Precision

Cosine Normalized SSIM PSNR MAE / 255 Max abs
flash CP2 vs efficient CP2 0.997295 0.969289 28.049 dB 4.750 / 255 196
flash CP2 vs SP2 0.997394 0.969862 28.226 dB 4.501 / 255 187
efficient CP2 vs SP2 0.997398 0.970382 28.218 dB 4.571 / 255 180

From tables above, we can conclude that comparing to efficient CP2, flash ring CP2 achieves comparable precision to ulysses SP2.

Throughput

mode Request wall time Denoise stage steady step
Flash CP2 2.946104 +/- 0.010519 s 2750.312 +/- 12.587 ms 550.423 +/- 2.351 ms
SP2 3.145325 +/- 0.011289 s 2951.521 +/- 12.741 ms 590.334 +/- 2.524 ms
Efficient CP2 3.459390 +/- 0.025342 s 3260.770 +/- 19.557 ms 652.187 +/- 4.197 ms

@JasonHe-WQ
JasonHe-WQ requested a review from akaitsuki-ii August 4, 2026 08:38
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant