返回全部动态
拓扑感知数据传输：面向分离式GPU推理的优化方案
原标题：Topology-Aware Data Movement for Disaggregated GPU Inference
AI 摘要
本文提出了一种面向分离式GPU推理的拓扑感知数据传输方案，解决预填充和解码分离时KV缓存传输的网络瓶颈问题。该方案通过流水线分层传输、NVLink域感知放置和CXL 3.0内存扩展器三种机制，相比统一RDMA可降低3至18倍传输延迟。目前仅提供分析模型和组件实现，尚需多节点异构集群和CXL 3.0硬件进行完整评估。
以上摘要由 AI 生成，可能存在误差。事实请以原文为准。
正文节选
Computer Science > Machine Learning Title:Topology-Aware Data Movement for Disaggregated GPU Inference View PDF HTML (experimental) Abstract:Disaggregated LLM inference creates a datacenter networking problem that no existing system solves correctly. When prefill and decode run on separate GPU pools, the KV cache must be transferred between them. For a 70B model this is 2.6 GB per request, exceeding 100 GB/s aggregate at production scale. Yet DistServe, Splitwise, and Mooncake all us
发布时间：2026-08-03 12:00
抓取时间：2026-08-03 15:21
来源机构：arXiv