SageMaker Inference 支持 NVIDIA MIG GPU 分区
原标题：SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
AI 摘要
AWS 宣布 SageMaker Inference 支持基于 NVIDIA MIG 的 GPU 分区，可将单块物理 GPU 划分为最多七个硬件隔离分区，每个分区拥有独立的计算单元与显存。该能力建立在 inference components 之上，允许在单个端点上部署比 GPU 数量更多的模型，并避免冷启动与相互干扰。Salesforce 参与了需求定义与初步验证，称该能力可提升 GPU 利用率并降低推理成本。
正文节选
SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints When you deploy a model to a GPU endpoint, the smallest unit of GPU you can request is one whole accelerator. This constraint can cost you twice. First, you can’t host more models on an instance than it has accelerators. An ml.p4d.24xlarge has eight A100s, so it holds eight models, whether each one needs 40 GB or 14 GB. Second, and less obvious, it makes every deployment come down to a choice of w