X-CoSD：通信高效的跨词表协作推测解码
原标题：X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding
AI 摘要
该论文提出 X-CoSD，一种面向设备端小模型与服务器大模型词表不一致场景的协作式推测解码框架。现有 CoSD 方法假设双方共享词表，且残差重采样需要交换词表上的概率分布，导致通信开销大。X-CoSD 通过混合重采样（HR）仅在公共词表区域传输分布，并进一步提出 X-CoSD-E，由服务器只发送替换候选及其概率供设备本地验证。论文证明两者均保持服务器 LLM 的分布，实验显示在保持生成质量的同时显著提升 token 生成速度。
正文节选
X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding Abstract This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device small language model (SLM) drafts candidate tokens and a server LLM verifies them. Existing CoSD methods assume a shared vocabulary between the SLM and the LLM and incur substantial communication load because residual resampling requires token distribution