用并发扫描为 SageMaker AI 生成式端点做容量规划
原标题：Right-size generative AI endpoints with concurrency sweeps on Amazon SageMaker AI
AI 摘要
AWS 博客介绍在 Amazon SageMaker AI 上通过并发扫描（concurrency sweep）为生成式 AI 端点做容量规划的方法。文章以部署 NVIDIA Nemotron-3 Nano 30B（MoE，仅 3B 激活参数）到 ml.g7e.2xlarge 为例，使用原生 vLLM 容器和 CreateAIBenchmarkJob API 逐步增加并发流量，测量吞吐与延迟曲线，找出饱和点、SLA 突破点和所需实例数。该功能内置于 SageMaker AI Inference Recommendations，无需自建压测基础设施。
正文节选
Right-size generative AI endpoints with concurrency sweeps on Amazon SageMaker AI Concurrency sweeps help you right-size a generative AI endpoint by finding the instance type and serving configuration that maximizes price-performance while holding latency within acceptable bounds. Without a systematic approach, right-sizing means deploying, load-testing manually, adjusting, and repeating until the numbers look acceptable. Choose five ml.g7e.2xlarge instances when one would suffice, and you burn