DISCO：通信高效的分布式投机解码框架
原标题：Distributed LLM Inference via Communication-Efficient Speculative Decoding
AI 摘要
论文提出 DISCO，一种面向设备-服务器协同 LLM 推理的通信高效分布式投机解码框架。现有方法在拒绝采样时需在设备与服务器间传输整个词表规模的 token 分布，带宽受限时开销很大。DISCO 在服务器端部署草稿小模型（SLM）的副本，使服务器本地即可获得拒绝采样所需的草稿侧分布，从而只需交换草稿 token 索引和标量概率值。实验表明，在多种带宽条件下，DISCO 的解码加速比均优于现有分布式投机解码方法，低带宽场景下提升尤为明显。
正文节选
Distributed LLM Inference via Communication-Efficient Speculative DecodingThanks: J. Shin and B. Park contributed equally to this work. Abstract Distributed speculative decoding has emerged as a promising approach for device–server collaborative large language model (LLM) inference, but its decoding speedup is often limited by the vocabulary-scale communication overhead. In existing distributed speculative decoding methods, exact rejection sampling requires transmitting full token distributions