ReplaySSM：重思状态存储以加速混合模型解码
原标题：ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding
AI 摘要
NVIDIA 与普林斯顿 Dao AI Lab 联合提出 ReplaySSM，将 SSM 状态表示为检查点加近期输入缓冲区，从而把输出计算与状态更新解耦。该方法针对混合模型解码中的状态流量、推测解码回滚和量化状态误差累积三大问题，在 vLLM 中实现并集成到 TensorRT-LLM、FlashInfer，已在 GLM-5.3-Flash 和 Kimi-K3 生产环境运行。
正文节选
Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are all built this way, with Mamba-2, Gated DeltaNet (GDN), or Kimi Delta Attention (KDA) as the SSM layer. SSM layers are designed to make decoding fast: they use a fixed-size state in place of a key-value (KV) cache that grows with context length. At serving batch sizes, however, the SSM layers, not attention, can be the latency bottleneck