Skip to content

[AMD][MI355X] kimik3-fp4-mi355x-vllm-agentic-mtp a4w4 MoE + lmcache 0.5.6 - #3447

Draft
seungrokj wants to merge 3 commits into
mainfrom
amd/agentx_k3_vllm_0925
Draft

seungrokj wants to merge 3 commits into
mainfrom
amd/agentx_k3_vllm_0925

Conversation

@seungrokj

Copy link
Copy Markdown
Collaborator

Summary

  • Switch kimik3-fp4-mi355x-vllm-agentic-mtp to the canonical VLLM_ROCM_USE_AITER_MOE_SITUV2=1 flag so the SiTUv2 a4w4 FlyDSL MoE path is selected unambiguously, and drop the deprecated AITER_SITUV2_A8W4 override.
  • Bump the vLLM ROCm image to nightly-rocm100-29468dde and LMCache to 0.5.6.dev101+rocm7.2.
  • Trim the search space to a single lmcache DRAM dcp-size 8 arm at conc [70, 80], dram-utilization 0.50.

Test plan

  • e2e run: test-config --config-files ./configs/amd-master.yaml --config-keys kimik3-fp4-mi355x-vllm-agentic-mtp --no-evals

🤖 Generated with Claude Code

Switch the MI355X Kimi-K3 vLLM recipe to the canonical
VLLM_ROCM_USE_AITER_MOE_SITUV2=1 flag so the SiTUv2 a4w4 FlyDSL MoE path
is selected unambiguously, and drop the legacy AITER_SITUV2_A8W4 override.
Bump the vLLM ROCm image to nightly-rocm100-29468dde and LMCache to
0.5.6.dev101+rocm7.2. Trim the search space to a single lmcache DRAM
dcp-size 8 arm at conc [70, 80] and dram-utilization 0.50.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast;仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

…e-sweep

Record the switch to the canonical VLLM_ROCM_USE_AITER_MOE_SITUV2=1 a4w4
FlyDSL MoE path, the nightly-rocm100-29468dde image, LMCache 0.5.6.dev101,
and the trimmed TP8/DCP8 lmcache conc [70, 80] search space for
kimik3-fp4-mi355x-vllm-agentic-mtp (PR #3447).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@seungrokj seungrokj changed the title config(kimik3): a4w4 MoE + lmcache 0.5.6 dcp8 re-sweep on MI355X vLLM [AMD] kimik3: a4w4 MoE + lmcache 0.5.6 dcp8 re-sweep on MI355X vLLM Sep 26, 2026
@seungrokj seungrokj changed the title [AMD] kimik3: a4w4 MoE + lmcache 0.5.6 dcp8 re-sweep on MI355X vLLM [AMD][MI355X] kimik3-fp4-mi355x-vllm-agentic-mtp a4w4 MoE + lmcache 0.5.6 Sep 26, 2026
@seungrokj seungrokj added AMD agentx AgentX benchmarks, recipes, and infrastructure labels Sep 26, 2026
Split the 8|10 concurrency branch so conc 8 uses SPEC_NUM_TOKENS=7 with
synthetic acceptance length 3.84; conc 10 keeps k=4 / AL 3.36.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@seungrokj

Copy link
Copy Markdown
Collaborator Author

Need LMC fix on rocm env.

@functionstackx

Copy link
Copy Markdown
Collaborator

InferenceX has switched away from unmaintainable bash scripts to YAML files that don't repeat the same stuff over and over again. Please merge the latest main into this PR: we have migrated single-node AgentX onto native srt-slurm (#3428), so AgentX configs are now declarative YAML recipes, not per-config 1000+ line bash slop scripts. Please also delete the old benchmarks/single_node/** scripts (see this recipe for the new format).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agentx AgentX benchmarks, recipes, and infrastructure AMD

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants