分层 Copula-Gumbel-Top-K 路由：冻结 MoE 的双侧依赖控制
原标题：Hierarchical Copula-Gumbel-Top-\texorpdfstring{$K$}{K} Routing: Two-Sided Dependence Control for Frozen Mixture-of-Experts at Fixed Per-Token Routing Laws
AI 摘要
该论文提出了一种名为 Hierarchical Copula-Gumbel-Top-K 的新型路由机制，用于冻结的混合专家模型，在不改变每个 token 路由分布的前提下，通过高斯 copula 和反演构造分别控制组内正相关和组间负相关，从而调节专家负载的方差。该方法通过小规模控制器在冻结特征上训练，仅需前向传播，初步实验验证了机制的有效性，但未展示任务级微调收益。
正文节选
Computer Science > Machine Learning Title:Hierarchical Copula-Gumbel-Top-\texorpdfstring{$K$}{K} Routing: Two-Sided Dependence Control for Frozen Mixture-of-Experts at Fixed Per-Token Routing Laws View PDF HTML (experimental) Abstract:A stochastic Gumbel-Top-$K$ router defines, for every token of a mixture-of-experts (MoE) model, a \emph{routing law}: a distribution over ordered expert lists and mixture weights. We ask which \emph{joint} distributions over the routing choices of diff