AWS 推出多节点 Ray LLM 深度学习容器，支持 EKS 上 PD 分离推理
原标题：Run multi-node inference workloads using Ray LLM Deep Learning Containers
AI 摘要
AWS 发布面向多节点推理的 Ray LLM Deep Learning Containers（DLC），用于在 Amazon EKS 上跨两个 GPU 节点部署单个大语言模型。该方案将 prefill 与 decode 阶段分离到不同节点，通过 NIXL 和 EFA 传输 KV cache，并对外暴露 OpenAI 兼容端点。容器内置 vLLM、Ray Serve LLM 等组件，无需编写应用代码，适合 RAG、文档摘要和 agentic 应用等场景。
正文节选
Run multi-node inference workloads using Ray LLM Deep Learning Containers This post shows how to deploy one large language model (LLM) across two GPU nodes on Amazon Elastic Kubernetes Service (Amazon EKS) with the multi-node Ray LLM Deep Learning Containers. It uses separate prefill and decode nodes behind one OpenAI-compatible endpoint. This kind of multi-node setup is common for tensor, pipeline, and data parallelism, as well as prefill/decode disaggregation inference workloads. In a previous