[TRTLLM-15177][chore] Kimi K3: split long weight loaders + test hygiene - #18976
Draft
brnguyen2 wants to merge 1 commit into
Draft
[TRTLLM-15177][chore] Kimi K3: split long weight loaders + test hygiene#18976brnguyen2 wants to merge 1 commit into
brnguyen2 wants to merge 1 commit into
Conversation
Deferred cleanup from PR NVIDIA#17269 review: - Split _load_trunk_params / _load_expert_slices into cohesive private helpers (code moved verbatim, no behavior change). - test_kimi_kda_fused_verify_parity: hoist the with_multi_stream import to the guarded module-level import block, and build the per-layer caches as real PythonMambaCacheManager.SpeculativeState objects instead of SimpleNamespace mocks. Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
Collaborator
Author
|
/bot run --disable-fail-fast |
Collaborator
|
PR_Github #72497 [ run ] triggered by Bot. Commit: |
Collaborator
|
PR_Github #72497 [ run ] completed with state
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[TRTLLM-15177][chore] Kimi K3: split long weight loaders + test hygiene
Description
Closes out TRTLLM-15177, the cleanup items deferred from the #17269 review. These are the last live items of that list — every other item either already landed on
mainindependently or is being dropped as will-not-do.Split the two long weight-loader methods in
tensorrt_llm/_torch/models/modeling_kimi_linear.py(review item 2.1)._load_trunk_params(~190 lines) and_load_expert_slices(~225 lines) are split into cohesive private helpers, with the code moved verbatim (no behavior change):_load_trunk_params→_build_mla_head_shard_maps,_kda_tp_shard_info,_load_gate_up_fused_param,_load_mla_kv_b_param,_slice_a_log_checkpoint_pad_load_expert_slices→_stream_expert_slices_by_file,_verify_expert_slots_filledThe nested closures that carry shared load state (
load_param,ensure_prepared,maybe_finalize_layer,load_expert) stay nested; lifting them would have meant threading lock/bookkeeping state through every signature for no readability gain.Hoist an in-function import in
tests/unittest/_torch/modules/kimi_kda/test_kimi_kda_fused_verify_parity.py(item 4.4):with_multi_streammoves from inside the test to the module-level guarded import block, preserving the skip-on-missing-dependency behavior.Use the real cache object in the parity test (item 4.1): the fused/sequential per-layer caches are now real
PythonMambaCacheManager.SpeculativeStateinstances instead oftypes.SimpleNamespacemocks, so the test exercises the class's actual field set and itshas_kda_replay_cachesproperty. The unused baseconv/temporalfields areNone(the test passes the pools toforward_verifydirectly, as production does).Test Coverage
Pure refactor plus test hygiene — no new code paths. The moved loader code is exercised unchanged by the existing Kimi K3 L0 suites;
test_kimi_kda_fused_verify_parity.pycovers the fused-verify path (requires Blackwell). Verified locally withpre-commit runon both files (all hooks pass) and AST/compile checks.PR Checklist
Please review the following before submitting your PR:
[JIRA ticket/NVBugs ID/GitHub issue/None][type] Summary of the change