Update B300 DeepSeek V4.1 Flash vLLM nightly / 更新 B300 DeepSeek V4.1 Flash vLLM nightly

[xinli-sw]

Pin the B300 DeepSeek-V4.1-Flash AgentX arm to vLLM nightly 0961bbae. Enable the merged Blackwell sparse indexer settings and select FlashInfer at TP2, MegaAttention at TP4. Preserve the existing topology and memory limits. Also pass the engine readiness timeout from the launcher where needed.

The vLLM tracking issue and upstream recipe document the included changes. Local shell, YAML, and exact-key matrix checks pass; GPU sweep and eval are pending.

中文

将 B300 DeepSeek-V4.1-Flash AgentX 配置固定到 vLLM nightly 0961bbae。启用已合并的 Blackwell 稀疏索引器参数，TP2 选择 FlashInfer，TP4 选择 MegaAttention；保留现有拓扑与显存限制，并由启动脚本传入所需的引擎就绪超时。

相关改动见 vLLM 跟踪问题 与上游配方。本地 shell、YAML 和指定配置的矩阵检查通过；GPU sweep 与 eval 待运行。

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

Claude finished @xinli-sw's task in 2m 51s —— View job

---

Review of PR #3394

• Gather context (diff vs origin/main, changed files)
• Validate master config + perf-changelog rules
• Review benchmark script, launcher, and vLLM flag usage
• Post review

LGTM - no blocking issues found.

Checks that passed by inspection:

• configs/nvidia-master.yaml change is paired with a perf-changelog.yaml entry, appended at the tail, with a matching config key and PR link. No append-only entries involved.
• Image vllm/vllm-openai:nightly-0961bbae… is a public Docker Hub reference; runners/launch_b300-dsxe.sh still imports it via enroot import … docker://.
• Dropping the in-script VLLM_ENGINE_READY_TIMEOUT_S default is safe: the workflow sources benchmarks/runtime_settings.sh (3600) before the launcher, and the launcher now exports 7200 in the dsv41flash vLLM/SGLang branch before srun --export=ALL, so check_env_vars always sees a value. This matches the existing pattern in dsv41flash_fp4_vllm_mtp.sh.
• No hardcoded --enable-expert-parallel; vllm serve args are on separate lines.

Not verified in this run: the sandbox blocked network access (and the vLLM source MCP server timed out), so I could not confirm that FLASHINFER_MLA_SPARSE_DSV41, FLASHMLA_MEGA_ATTN_DSV41, and indexer_sparse_logits exist at commit 0961bbae, nor that the Docker Hub tag is published. The pending GPU sweep will exercise those directly.

[claude]

I reviewed this PR and didn't find any bugs. Because configs/nvidia-master.yaml is CODEOWNERS-protected and this change pins a new vLLM nightly image together with new attention/KV-cache flags on the launch path, a human (owner) look would still be worthwhile.

What was reviewed:

• Traced EVAL_ONLY / VLLM_ENGINE_READY_TIMEOUT_S through benchmarks/runtime_settings.sh, runners/launch_b300-dsxe.sh, and the GitHub workflow templates — every caller path already supplies both explicitly, so the new check_env_vars calls won't hard-fail.
• Verified the new TP2/TP4 --attention-config branch against configs/nvidia-master.yaml's search-space, which only defines tp: 2 and tp: 4 for this recipe.
• Checked the new indexer_kv_dtype":"mxfp4" / indexer_sparse_logits settings against CONTRIBUTING.md's draft-precision rule — these configure the target model's sparse-attention indexer (long-context KV selection), not the DSpark draft model, so that rule doesn't apply.
• Confirmed the perf-changelog.yaml entry is append-only and matches the described change.

Extended reasoning...

The diff bumps a pinned vLLM image tag, adds a TP-conditional --attention-config and --kv-cache-dtype fp8 to one benchmark launch script, hardens two env vars (EVAL_ONLY, VLLM_ENGINE_READY_TIMEOUT_S) to required check_env_vars instead of inline defaults, and appends a matching perf-changelog entry. It touches no auth, crypto, or data-exposure surface; this is pure benchmark/launch configuration. I independently traced the env-var propagation across all runner and workflow callers and confirmed no path would hit the new hard-fail, and checked the mxfp4/sparse-logits additions against the repo's draft-precision policy (they belong to the target model's sparse attention indexer, not the DSpark draft, so no violation). The deciding factor for defer over approve is that configs/nvidia-master.yaml falls under CODEOWNERS (@ ankur-singh, @ kedarpotdar-nv, @ SemiAnalysisAI/core), so a human owner sign-off is still warranted despite no findings.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36084679196

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36084679196

[claude]

Claude finished @xinli-sw's task in 2m 13s —— View job

---

Review of PR #3394 (ready for review)

• Gather context (diff vs origin/main, changed files)
• Check sweep / CI results since the last review
• Re-verify master config + perf-changelog rules and script flags
• Post review

LGTM - no blocking issues found. This is the same commit the two earlier Claude passes reviewed, so I am not repeating that feedback. Two things are new since then.

Sweep status. The first sweep (run 36065953173) finished and published unofficial-run links, and a fresh sweep (run 36066605303) started when the PR went ready for review. I could not read per-job conclusions from this sandbox, so please confirm both TP2 and TP4 arms are green in the new run before merge.

Description vs. diff. The PR text says the change preserves "the existing topology and memory limits", but the diff also drops the explicit --compilation-config capture-size list and every --max-num-batched-tokens tier (2048/8192), falling back to vLLM defaults, and replaces them with the B200-style doubling loop from DSV41_MIN_CUDAGRAPH_CAPTURE_SIZE (capped at 512 on TP2). That mirrors dsv41flash_fp4_vllm_mtp.sh and is not a bug, but it is a real scheduler-memory change on TP2 that the description does not mention. Worth a one-line update to the PR body and changelog so the curve delta is attributable.

Verified locally:

• check_env_vars DSV41_MIN_CUDAGRAPH_CAPTURE_SIZE EVAL_ONLY VLLM_ENGINE_READY_TIMEOUT_S is satisfied on every path: the launcher's dsv41flash vLLM/SGLang branch exports the first and last (runners/launch_b300-dsxe.sh:381-388), and EVAL_ONLY is set at workflow env level in benchmark-tmpl.yml, profile.yml, and speedbench-al.yml.
• perf-changelog.yaml entry is appended at the tail, config key matches, PR link present, no append-only entries involved. The schema only requires one description line, so the English-only entry validates.
• Image is a public Docker Hub reference and the launcher still runs import_squash_image on it.

Not verified (network blocked, vLLM source MCP server timed out): that FLASHINFER_MLA_SPARSE_DSV41, FLASHMLA_MEGA_ATTN_DSV41, indexer_kv_dtype: mxfp4, and indexer_sparse_logits are accepted at 0961bbae, that --kv-cache-dtype fp8 is compatible with both backends, and that the Docker tag is published. The running sweep exercises all of these directly.

[claude]

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline nit, I checked two other candidates around the new CAPTURE_SIZE doubling loop: a non-positive DSV41_MIN_CUDAGRAPH_CAPTURE_SIZE spinning forever, and the TP2/CONC=128 case where the 512 cap sits below CONC*(1+NUM_SPEC_TOKENS). Both mirror the pre-existing doubling-loop and 512 cap already used in benchmarks/single_node/agentic/dsv41flash_fp4_vllm_mtp.sh, so neither is a risk newly introduced by this change.

Extended reasoning...

This PR pins a new vLLM nightly and reworks CUDA-graph capture sizing plus attention-backend/KV-dtype selection in the B300 DeepSeek-V4.1-Flash vLLM benchmark script, adds a launcher env export, and appends a perf-changelog entry; it touches no auth/crypto/security surface. The confirmed inline finding is a stale doc comment in configs/nvidia-master.yaml. Two other candidate issues (an env-var-driven infinite loop and an undersized CUDA-graph cap at TP2/high concurrency) were investigated and ruled out because they replicate an already-shipped pattern in the sibling dsv41flash_fp4_vllm_mtp.sh script rather than being newly introduced.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

• 🟡 configs/nvidia-master.yaml — nit: the comment at configs/nvidia-master.yaml:8429-8430 ("The B300 script uses explicit piecewise CUDA graph capture tiers") is now stale. The diff replaced the piecewise GRAPH_SIZES/COMPILATION_CONFIG list in dsv41flash_fp4_b300_vllm_mtp.sh with a single doubling CAPTURE_SIZE, so there are no more capture tiers. Fix: update or remove the comment so it matches the new CAPTURE_SIZE logic.

  Why this was flagged

  configs/nvidia-master.yaml:8430 still says the B300 script uses explicit piecewise CUDA graph capture tiers, but benchmarks/single_node/agentic/dsv41flash_fp4_b300_vllm_mtp.sh no longer has GRAPH_SIZES/COMPILATION_CONFIG (removed by this diff) and instead derives a single CAPTURE_SIZE via a doubling loop. A maintainer reading the config comment gets a false description of the current script behavior. This is a documentation-only nit, not a functional bug.

  Verification: nit: The comment at configs/nvidia-master.yaml:8429-8430 ("The B300 script uses explicit piecewise CUDA graph capture tiers.") is now stale. This diff removed the piecewise machinery from benchmarks/single_node/agentic/dsv41flash_fp4_b300_vllm_mtp.sh — the diff deletes GRAPH_SIZES_2046='6,12,18,...,2046', GRAPH_SIZES_8190, CAPTURE_SIZE="${GRAPH_SIZES##*,}",… | nit: The comment at…

[functionstackx]

InferenceX has switched away from unmaintainable bash scripts to YAML files that don't repeat the same stuff over and over again. Please merge the latest main into this PR: we have migrated single-node AgentX onto native srt-slurm (#3428), so AgentX configs are now declarative YAML recipes, not per-config 1000+ line bash slop scripts. Please also delete the old benchmarks/single_node/** scripts (see this recipe for the new format).

[functionstackx]

Closing in favor of #3458, which ports this PR onto the srt-slurm recipe that #3428 switched dsv41flash-fp4-b300-vllm-agentic-dspark to: vLLM nightly-0961bbae, FlashInfer sparse attention with MXFP4 indexer KV and sparse logits at TP2 and TP4, fp8 KV cache, and a 7200 s engine readiness timeout. The legacy dsv41flash_fp4_b300_vllm_mtp.sh script this PR edits is no longer used by that config. Note: the description here mentions MegaAttention at TP4 but the script used FlashInfer at both TP sizes; #3458 follows the script. Thanks!