教师门控在线蒸馏：提示级验证提升训练效率与准确性
原标题：Paper page - Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation
AI 摘要
Hugging Face 每日论文页面介绍了一篇题为《Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation》的论文。该论文提出教师门控在线蒸馏（TGOPD）方法，通过验证器评分的探针在提示级别验证教师可靠性，并将提示路由到密集蒸馏或基于验证器的强化学习。实验表明，在数学、代码和指令跟随任务中，TGOPD 在 4B 和 35B 学生模型上均优于普通在线蒸馏，并将教师节点 GPU 利用率从 9.8% 提升至 78.9%。
正文节选
Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation Abstract Teacher-Gated On-Policy Distillation verifies teacher reliability per prompt via verifier-scored probes, routing to dense distillation or verifier-grounded reinforcement learning to improve training efficiency and accuracy. On-policy distillation (OPD) accelerates post-training by providing dense token-level supervision from a frozen teacher on the student's own rollouts. Vanilla OPD applies this supervisi