TreeGraft：基于树形投机解码的自适应多草稿模型嫁接方法
原标题：TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding
AI 摘要
TreeGraft 提出了一种多草稿模型框架，通过将不同成本的草稿模型（小型和中等）联合构建共享草稿树，以提升树形投机解码的效率。该方法利用较强模型对较弱模型的候选进行重打分、重新选择嫁接位置并恢复被忽略的路径，同时引入轻量级调度器控制调用强模型的时机。在10个模型对和6个基准测试中，TreeGraft 相比最优的单草稿模型策略平均提升15.1%，最高达26.6%。
正文节选
TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding Abstract Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing proposals into multiple candidate paths, increasing the accepted length. However, existing tree-structured methods use a single drafter for all drafting steps, creating a dilemma: a smaller drafter is fast but yields lower-quality trees