OMP-MoE：基于正交匹配追踪的 MoE 大模型高效专家剪枝
原标题：OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit
AI 摘要
该论文提出 OMP-MoE，一种无需训练的 MoE 大模型专家剪枝框架，将专家剪枝重构为稀疏信号重建问题，用正交匹配追踪（OMP）贪心选择专家，把搜索复杂度从指数级降到线性。方法还引入注水策略进行跨层专家预算分配，并提出自适应推理机制 OMP-MoE†。在 Qwen、DeepSeek-V2、GPT-OSS、Mixtral 上，25-50% 剪枝率下性能一致优于现有方法；Qwen3-30B-A3B 剪枝 50% 时保留 93.3% 原性能，搜索提速 33 倍，推理提速 1.55 倍。
正文节选
OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit Abstract Mixture-of-Experts (MoE) models enable efficient scaling of large language models but face critical deployment challenges due to massive memory requirements. Existing pruning methods either incur prohibitive search costs or neglect the dynamic interdependencies between experts. To address these challenges, we present OMP-MoE, a novel training-free compression framework for reducing expert redun