vLLM 双向 KV 传输优化多轮 Agent 的 TTFT
原标题：Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLM
AI 摘要
AWS 在 vLLM 中实现了双向 KV cache 传输机制，让 decode 节点在后续对话轮次中把已计算的 KV 块回传给 prefill 节点，避免多轮 agent 工作负载重复计算历史上下文。该功能最初在 AWS Trainium 集群上构建并贡献给 vLLM 作为与加速器无关的能力，在 EC2 P5en GPU 集群配合 EFA 网络的 1P1D 部署测试中，Qwen3-32B 的 TTFT 最高降低 1.6 倍，GLM-5-FP8 最高降低 3.3 倍。
正文节选
Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLM Multi-turn agentic workloads build on accumulated context as an agent reasons, invokes tools, and incorporates earlier results. With each new turn, the client resends the conversation history, including system instructions, prior prompts, tool results, and model responses, causing the growing context to be processed repeatedly. In Prefill-decode (P-D) disaggregation, each request is split across specialized nod