Skip to content

Fix logits_to_keep support in Qwen2-VL forwards - #1319

Open
Terminator666666 wants to merge 3 commits into
linkedin:mainfrom
Terminator666666:fix/qwen2-vl-logits-to-keep
Open

Fix logits_to_keep support in Qwen2-VL forwards#1319
Terminator666666 wants to merge 3 commits into
linkedin:mainfrom
Terminator666666:fix/qwen2-vl-logits-to-keep

Conversation

@Terminator666666

Copy link
Copy Markdown

Summary

Fix logits_to_keep support in the Liger forward replacements for Qwen2-VL and Qwen2.5-VL.

The argument was previously captured by **kwargs, forwarded to the base multimodal model, and ignored when computing logits. The patched forwards now explicitly consume it and select the requested hidden-state positions before lm_head, matching Transformers 5.8.1 behavior.

Details

  • Support both integer token counts and tensor indices.
  • Preserve the default logits_to_keep=0 all-token behavior.
  • Keep the fused-loss path on the complete hidden-state tensor.
  • Add focused CPU tests for both model families, including argument forwarding and fused-loss coverage.

Testing Done

  • Hardware Type: NVIDIA GeForce RTX 5070 Laptop GPU (tests used CPU tensors only)
  • run make test to ensure correctness
  • run make checkstyle to ensure code style
  • run make test-convergence to ensure convergence

Targeted tests:

python -m pytest test/transformers/test_qwen2_vl_forward.py -q --confcutdir=test/transformers -p no:cacheprovider
11 passed

python -m pytest test/transformers/test_monkey_patch.py -k "qwen2_vl or qwen2_5_vl" -q -p no:cacheprovider
6 passed, 56 deselected

make checkstyle
All checks passed
317 files left unchanged

The full test and convergence suites were not run locally because this is a forward-compatibility fix validated with small CPU tensors; upstream CI can provide the broader GPU coverage.

@Terminator666666

Copy link
Copy Markdown
Author

????(WSL Ubuntu,RTX 5070 Laptop GPU,BF16,?? test/convergence/bf16/test_mini_models_with_logits.py):

  • mini_qwen2_5_vl:?? 32 ? + Liger 32 ?,loss?top-k logprobs?????????,1 passed,28 ??
  • mini_qwen2_vl:?? 32 ? + Liger 32 ?,loss?top-k logprobs?????????,1 passed,29 ??

????????? tiny Shakespeare(batch size 16,sequence length 128),??????????

?????:11 ? CPU forward ?????6 ? monkey-patch ?????Ruff/checkstyle/py_compile/git diff --check ????

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant