Add Less is MoE (EMNLP 2026) to Mixture-of-Experts(MoE) LLM Inference

[HectorHHZ]

Adding one paper to Mixture-of-Experts(MoE) LLM Inference. Disclosure: I am an author.

• Less is MoE: Trimming Experts in Domain-Specialist Language Models — EMNLP 2026 (Main Conference, Oral)
• Paper: https://arxiv.org/abs/2606.05538
• Code: https://github.com/HectorHHZ/Less-is-MoE

Why it fits. Most MoE compression drops or merges whole experts. This paper shows capability is concentrated in a tiny number of FFN intermediate dimensions — on Qwen1.5-MoE, removing 12 of 1.35M routed-FFN dimensions collapses GSM8K — and prunes at that granularity instead. At a 50% compression ratio it cuts weight memory by ~45% and improves inference throughput by 21%, so the result is an inference-side win, not only a parameter-count one. Evaluated on Qwen1.5-MoE, Qwen3-MoE and OLMoE, with vLLM and Hugging Face runtime patches in the repo.

Followed the conventions of the table: appended at the end of the MoE section in ascending date order, [[pdf]] links to the arXiv PDF, and the code link carries the usual stars badge.