AutoJudge：自动化数据集筛选加速 LLM 推理

AI 摘要

Together AI 推出 AutoJudge，一种通过自动化数据集筛选加速大语言模型推理的方法。它采用有损投机解码，无需人工标注，可接受最多 40 个草稿令牌，实现 1.5-2 倍加速，并兼容 vLLM 等主流框架。研究将在 NeurIPS 2025 展示。

正文节选

We introduce AutoJudge, a method that accelerates large language model (LLM) inference through task-specific lossy speculative decoding. Instead of matching the target model’s output distribution token by token, this method identifies which specific generated tokens affect downstream quality. Compared to prior approaches, AutoJudge does not require manual annotation, as it employs a classifier trained in a self-supervised manner. AutoJudge can accept up to 40 draft tokens per verification cycle

发布时间：—

抓取时间：2026-08-03 01:13

来源机构：Together AI