NVIDIA TensorRT 多设备推理集成 Dynamo-Triton，跨多 GPU 服务模型
原标题：Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton
AI 摘要
NVIDIA 发布 TensorRT 多设备推理能力，使单个 TensorRT 网络可借助 NCCL 分布式集合通信跨多 GPU 执行，并从 TensorRT 11.0 起全面支持。Dynamo-Triton 26.07 版本的 TensorRT 后端启用该能力，一个 KIND_MODEL 实例可拥有多块 GPU 并统一暴露单个 gRPC 模型端点。文章以 Cosmos 3 Nano 视频生成模型为例，端到端延迟从单 GPU 的 156.595 秒降至 8 GPU 的 34.183 秒，加速 4.58 倍，transformer RPC 加速达 6.09 倍。
正文节选
The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability that enables a single TensorRT network to execute across multiple GPUs using NCCL-backed distributed collectives while retaining TensorRT inference optimizations. It is fully supported starting with TensorRT 11.0. NVIDIA Dynamo-Triton (formerly NVIDIA Triton Inference Server) release 26.07 enables the multi-device inference capability of th