TEXAS：面向下游 MoE 大模型适配的任务专家感知监督方法
原标题：TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation
AI 摘要
arXiv 论文提出 TEXAS 方法，用于混合专家（MoE）大语言模型的下游适配。该方法通过对比成功与失败实例上的专家激活，识别任务相关专家，并在微调时对失败实例中激活这些专家的答案 token 赋予更高权重。在三个 MoE 模型和六个基准上，TEXAS 在 18 个设置中的 17 个达到最佳或并列最佳，平均比最强基线提升 1.3-1.5 分。
正文节选
Computer Science > Computation and Language Title:TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation View PDF HTML (experimental) Abstract:Mixture-of-Experts (MoE) language models route each token through a small subset of experts, making routing patterns useful for identifying task-relevant experts during downstream adaptation. Yet current approaches have two limitations: task experts are typically identified from aggregate routing statistics that