Meta Expert：面向存储与检索高效 MoE 的跨层专家共享
原标题：Paper page - Meta Expert: Cross-Layer Expert Sharing for Storage- and Retrieval-Efficient MoE
AI 摘要
该论文提出 Meta Expert 方法，通过在 MoE 模型的不同层之间共享元专家，减少层间专家的冗余存储。实证对比显示，在激活专家数量和专家粒度固定的条件下，Meta Expert 模型仅用 70–80% 的专家数量即可达到 Vanilla MoE 的性能，且模型越深越大节省越明显。该方法已在 Intern-S2-Mobius 中完成规模化验证，并开源了代码与模型权重。
正文节选
Meta Expert: Cross-Layer Expert Sharing for Storage- and Retrieval-Efficient MoE Abstract Shared meta-experts across layers reduce redundancy and improve efficiency compared to layer-specific local experts in mixture-of-experts models. As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling. However, mainstream models currently adopt hierarchical architectures, where experts in different layers, although topologically independe