NVIDIA 开源 NVCRE：在 AI 负载上线前验证 GPU 集群就绪
原标题：Validate GPU Cluster Readiness Before AI Workloads Land
AI 摘要
NVIDIA 发布开源 Kubernetes 控制器 NVCRE（Cluster Readiness Engine），用于在 AI 生产负载上线前验证 GPU 集群就绪状态。它通过运行真实的分布式工作负载（如 NCCL 通信测试、DCGM 诊断、NeMo 预训练）并按拓扑感知分组测量结果，将失败精确定位到具体节点和类别。该工具填补了 Kubernetes 缺乏类似 Slurm srun 一键测试能力的空白，使集群就绪从假设变为可证明的属性。
正文节选
A GPU cluster can pass every health check and still fail to run an AI workload. Even when every GPU, network link, and pod reports healthy, a 512-GPU training job can underperform or fail. The cause may be one slow GPU, a link that degrades under load, or a configuration that quietly routes traffic over a slower path. Operators may not discover the problem until hours into the run or until a customer files a ticket. Teams can then spend days bisecting the cluster to find the root cause while the