vLLM 推出基于 push 的 KV 传输优化分离式推理
原标题：Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
AI 摘要
AWS 机器学习博客介绍 vLLM 中基于 push 的 KV 传输方案，用于分离式推理（prefill/decode 分离）。该方案将 prefill 后的传输协调与设置移出关键路径，在 AWS EC2 P5en 实例配合 EFA 的 1P1D 部署上，对 Qwen3-32B 模型实现最高 3 倍 TTFT 改善和最高 30% TPOT 改善。该功能先在 AWS Trainium 集群上构建验证，后作为加速器无关特性贡献给 vLLM。
正文节选
Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM vLLM is an open-source inference and serving engine for large language models (LLMs). LLM inference has two distinct phases: prefill, which processes the input prompt and builds the key-value (KV) cache, and decode, which uses that cache to generate output one token at a time. These phases have different compute, memory, and scaling characteristics. Prefill-decode (P-D) disaggregation runs them on separate engine