DAS框架：分布感知投机解码加速RL训练rollout高达50%
原标题：Distribution-aware speculative decoding (DAS) is a novel framework that significantly alleviates the rollout bottleneck in RL post-training — delivering up to 50% speedup without touching model output
AI 摘要
Together AI 博客介绍了分布感知投机解码（DAS）框架，用于加速强化学习（RL）后训练中的 rollout 阶段，最高可提速 50% 且不改变模型输出。DAS 通过自适应后缀树草稿模型和长度感知调度策略，解决了长尾生成导致的 GPU 利用率低和同步瓶颈问题。实验表明，在数学推理和代码生成任务上，DAS 分别实现了超过 50% 和约 25% 的 rollout 时间减少，同时保持奖励质量不变。
正文节选
Distribution-aware speculative decoding (DAS) is a novel framework that significantly alleviates the rollout bottleneck in RL post-training — delivering up to 50% speedup without touching model outputs. The rollout bottleneck Reinforcement learning has become the cornerstone of modern LLM post-training. Models like DeepSeek-R1 owe their reasoning capabilities to RL fine-tuning. But as models grow larger, a critical bottleneck has emerged: the rollout phase. In RL training, the model must generat