Repository navigation
[Attention] Lazy-load LLM and ViT attention backends - #1668
Conversation
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Backend import contract lightllm/common/basemodel/attention/import_utils.py, lightllm/common/basemodel/attention/__init__.py, lightllm/common/basemodel/attention/linear/__init__.py, lightllm/common/basemodel/attention/nsa/__init__.py, lightllm/models/qwen3_dflash/model.py |
Added load_backend_class to resolve module-and-class paths. Removed package-level backend re-exports and changed the Qwen model to import FA3 classes from their backend modules. |
Attention backend selection lightllm/common/basemodel/attention/create_utils.py, lightllm/common/basemodel/attention/create_linear_utils.py, lightllm/common/basemodel/attention/nsa/dsv4_fp8_flashmla_sparse.py, unit_tests/common/basemodel/attention/test_lazy_import.py, unit_tests/common/basemodel/attention/linear/test_gdn.py |
Regular, MLA, NSA, Neo, and linear backend maps now use import paths. Selectors load configured, validated, or fallback backends when selected. Tests cover selection order, import isolation, and fallback behavior. |
ViT backend selection lightllm/common/basemodel/attention_vit/create_utils.py, unit_tests/common/basemodel/attention/test_lazy_import.py |
ViT backend maps now use import paths, and the selected class is loaded during selection. Tests cover explicit and automatic selection. |
Priority:
Estimated code review effort: 3 (Moderate) | ~20 minutes
Change: Refactor
Sequence Diagram(s)
sequenceDiagram
participant PrefillSelector
participant _auto_select_backend
participant load_backend_class
participant importlib
PrefillSelector->>_auto_select_backend: Validate candidates in priority order
_auto_select_backend->>load_backend_class: Load selected import path
load_backend_class->>importlib: Import module relative to attention package
importlib-->>load_backend_class: Return imported module
load_backend_class-->>_auto_select_backend: Return backend class
_auto_select_backend-->>PrefillSelector: Return selected backend class
Suggested reviewers: sufubao, sangchengc, wandy666
Merge Risk: 
Minimal · up to 2c4b9
-
No actionable issue was established in the changed backend-loading paths. The PR is ready to merge after normal checks.
Pre-merge checks: 
4 · 
1
-
Failed checks: 1 warning Passed checks: 4 passedFull details: Docstring Coverage
-
Explanation
Docstring coverage is 6.25% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 16 functions across 8 files.
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
-
Finishing Touches: 💡 1
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
No description provided.