LRMoE：面向显存受限节点的 MoE 训练需求侧负载重校准
原标题：LRMoE: Demand-Side Load Re-Calibration for Efficient MoE Training on Memory-Constrained Nodes
AI 摘要
该论文提出 LRMoE，一种面向显存受限节点的 MoE 训练系统，通过在前向阶段规划专家保留配额与放置、在反向传播阶段基于负载重校准指标（LRM）选择对专家更新贡献最大的 token-expert 对，并结合容量感知路由与物理压缩减少通信与计算开销。在 Qwen3-30B-A3B 和 GLM4.5-106B-A12B 上，相比 SYMI 将 MoE 层训练至固定损失目标的时间缩短 21.2%-44.8%。
正文节选
LRMoE: Demand-Side Load Re-Calibration for Efficient MoE Training on Memory-Constrained Nodes Abstract. Sparse Mixture-of-Experts (MoE) models offer scalable capacity for Web applications, but uneven routing creates computation and communication bottlenecks. Existing placement and replication methods redistribute this workload, yet additional expert copies compete with token-related buffers for limited GPU memory. We present LRMoE, a memory-aware MoE training system that jointly recalibrates bac