量化MoE中路由翻转检测易但修复判断难
原标题：Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Experts
AI 摘要
本研究针对量化混合专家模型（MoE）中的路由翻转问题，提出了一种因果分析框架，并发现检测路由翻转比判断其是否需要修复更容易。实验表明，在4位KV缓存量化下，约三分之一的性能损失由路由中介引起，但现有推理可观测的统计量无法预测翻转的损失符号，存在经验性的收益检测障碍。研究还发现参考保真度的修复效果因架构而异，且门控归一化约定仅影响损伤幅度而非路由可恢复性。
正文节选
Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Experts Abstract Top- Mixture-of-Experts (MoE) routing is discontinuous, so a deployment-motivated numerical disturbance—simulated 4-bit KV-cache quantization read by a protected BF16 gate—pushes tokens across decision boundaries and flips which experts fire. This paper proposes no new mitigation; it supplies a causal apparatus, several empirical findings, and a detection-limit r