[Feature] Add LFM2, LFM2-MoE, and LFM2-VL support

[alay2shah]

Summary

Adds Liger Kernel support for the Liquid AI LFM2 model family in Hugging Face Transformers:

• LFM2ForCausalLM
• LFM2MoeForCausalLM
• Lfm2VlForConditionalGeneration

The integration adds custom Triton kernels for the gated short convolution and MoE router, integrates LFM2-MoE expert tensors with the Liger fused MoE path, patches the SigLIP2 vision tower where a Liger implementation is beneficial, and supports fused linear cross entropy. It is exposed through the normal AutoLigerKernelForCausalLM and monkey-patch APIs, so downstream trainers only need the correct Liger version and use_liger_kernel=True.

Kernel changes

Gated short convolution

• Fuses the LFM2 B/C gates with causal depthwise convolution.
• Processes all K taps for one hidden channel in one weight-backward program, reusing gradients and gate values and reducing the launch grid by K.
• Avoids bias partials for the released bias-free LFM2 configurations.
• Uses portable 512-element forward/input-backward blocks and backend-aware weight-backward dispatch, including Hopper long-sequence configurations.

MoE router and experts

• Fuses sigmoid, top-k selection, optional normalization, and routing scale.
• Writes the complete dense router gradient from Triton, eliminating a separate zero-fill launch.
• Uses workload-aware expert dispatch on both MI325X and H100. The portable Liger fused Triton experts win at smaller routed-row counts; grouped MM wins above the crossover.
• Persists Triton autotune selections when the installed Triton supports cache_results, while remaining compatible with older Triton releases.
• Uses HIP-safe fixed configurations for LIGER_FUSED_MOE_AUTOTUNE=0; CUDA retains the existing fixed configurations.

ROCm fused linear cross entropy

• Raises the logits chunk floor for large hidden states on HIP, capped at 128 MiB, to avoid dozens of tiny GEMMs.
• Defaults the LFM2 fused linear cross-entropy replacement on for HIP and off for CUDA; callers can explicitly override it. The H >= 1024 guard preserves upstream numerical behavior for mini models.

LFM2-VL / SigLIP2

• Patches the text decoder and vision tower through the normal LFM2-VL API.
• Defaults Liger LayerNorm off on both backends because native LayerNorm is faster for the released SigLIP2 shape; callers can explicitly request it.
• A custom SigLIP GELU kernel was evaluated and rejected because native PyTorch tanh-GELU was faster on MI325X.

Kernel benchmarks

Post-H100-update AMD rerun on exact commit de8993f82af07e7fb9025ae6e506f2db14851266. Hardware: AMD Instinct MI325X, BF16. Values are median full forward+backward latency.

Kernel / shape	Reference	Selected path	Result
Short conv, T=4,096, H=2,048	0.826 ms	Liger 0.463 ms	1.79x faster, 31.1% less peak allocation
Short conv, T=65,536, H=2,048	20.583 ms	Liger 8.083 ms	2.55x faster, 31.1% less peak allocation
MoE router, T=4,096, E=32, K=4	0.107 ms	Liger 0.036 ms	2.99x faster, 36.5% less peak allocation
MoE router, T=65,536, E=32, K=4	0.329 ms	Liger 0.209 ms	1.57x faster, 36.5% less peak allocation
8B experts, T=512, E=32, K=4	grouped MM 2.844 ms	Liger 1.869 ms	Liger 1.52x faster
8B experts, T=4,096, E=32, K=4	Liger 6.066 ms	grouped MM 5.458 ms	grouped MM 1.11x faster
24B experts, T=2,048, E=64, K=4	grouped MM 5.716 ms	Liger 3.900 ms	Liger 1.47x faster
24B experts, T=4,096, E=64, K=4	grouped MM 7.228 ms	Liger 5.979 ms	Liger 1.21x faster
24B experts, T=8,192, E=64, K=4	Liger 10.336 ms	grouped MM 9.668 ms	grouped MM 1.07x faster

The expert table compares the two production paths selected by the workload-aware dispatch, not the generic Python expert loop. H100 kernel benchmark rows are committed in benchmark/data/all_benchmark_data.csv; this table is the independent MI325X rerun after the H100 changes landed.

End-to-end training

Post-H100-update rerun through leap-finetune on MI325X with BF16. Each workload used counterbalanced baseline/Liger/Liger/baseline runs on an exclusive node. Throughput is the mean of each pair's inverse median synchronized optimizer-step time after five warmup steps. Memory is the Trainer transient per-rank peak allocation delta; for two GPUs the maximum rank is reported.

Workload	Baseline steps/s	Liger steps/s	Throughput	Transient peak memory
LFM2 SFT, 4K, 1 GPU	7.692	9.762	+26.9%	-29.6%
LFM2 SFT, 4K, 2 GPUs	3.755	4.136	+10.2%	-26.7%
LFM2 DPO, 1 GPU	4.861	8.101	+66.6%	-33.5%
LFM2 DPO, 2 GPUs	3.567	4.272	+19.8%	-26.0%
LFM2 text GRPO, 1 GPU	2.373	2.832	+19.4%	-9.2%
LFM2 text GRPO, 2 GPUs	1.764	1.971	+11.7%	-0.5%
LFM2-MoE SFT, 4K, 1 GPU	1.803	2.022	+12.2%	-0.7%
LFM2-MoE SFT, 4K, 2 GPUs	1.035	1.081	+4.5%	-1.2%
LFM2-VL SFT, 4K text, 1 GPU	6.023	6.440	+6.9%	-36.0%
LFM2-VL SFT, 4K text, 2 GPUs	2.697	2.681	-0.6%	-27.4%
LFM2-VL GRPO, 1 GPU	0.909	0.965	+6.1%	-9.7%
LFM2-VL GRPO, 2 GPUs	0.793	0.813	+2.5%	-0.2%

Dense SFT and DPO benefit most because the optimized language-model path is a large fraction of total step time. MoE memory remains nearly flat because expert parameters and optimizer/FSDP state dominate the full-model footprint. VL and GRPO include vision, generation, reward, and communication work that Liger does not optimize, so their gains are smaller. The VL 2-GPU throughput result is effectively neutral within run-to-run variance while retaining a 27.4% transient peak-memory reduction.

Cold Triton compilation is excluded by the synchronized warmup/median methodology; short ephemeral jobs still need to account for cache fill.

Testing done

• Hardware Type: AMD Instinct MI325X
• make test: 3,931 passed, 1,184 skipped, 14 xfailed on the LFM2 feature branch
• Ruff check/format and git diff --check pass for the final changed files
• Post-H100-squash AMD focused suite: 46 passed (instance patching, backend defaults, MoE dispatch/router, and short conv)
• Post-H100-squash LFM convergence: all 6 FP32/BF16 cases pass for LFM2, LFM2-MoE, and LFM2-VL
• Post-H100-squash single-/two-GPU training for dense LFM2, LFM2-MoE, LFM2-VL, DPO, text GRPO, and VL-GRPO
• Four counterbalanced timing records per workload; all 12 SLURM jobs exited 0
• MoE 8B-A1B and 24B-A2B expert-shape sweeps
• H100 kernel benchmark data committed separately in the benchmark CSV