PRIME：面向MoE压缩的频率感知统一剪枝与合并

AI 摘要

该论文提出 PRIME，一个面向混合专家（MoE）模型的频率感知统一剪枝与合并压缩框架。PRIME 将离散的剪枝-合并分配问题建模为基于频率加权代理目标的全局整数线性规划（ILP），联合优化剪枝与合并决策，并通过误差界分析论证高频专家更适合作为聚类中心。在 Qwen3-30B-A3B 和 Qwen3.5-122B-A10B 上的实验表明，PRIME 在知识、编程、数学和长上下文任务上保持较高精度，且在 vLLM 上量化了部署收益：专家数减少 25% 可降低权重内存 23.7%、扩大 KV 缓存 2.20 倍、提升输出吞吐 1.74 倍。

正文节选

PRIME: Frequency-Aware Unified Pruning and Merging for MoE Compression Abstract Mixture-of-Experts (MoE) models have emerged as an effective paradigm for scaling large language models, yet their massive parameter footprint poses severe memory challenges for deployment. Existing compression methods typically treat expert pruning and merging as disjoint operations and rely on greedy heuristics, often leading to irreversible information loss or representational pollution. In this paper, we propose

发布时间：2026-10-14 12:00

抓取时间：2026-10-14 14:16

来源机构：arXiv