AWS 与 NVIDIA 合作：利用 MPS 将 ASR 推理成本降低 75%
原标题：Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2
AI 摘要
AWS、NVIDIA 与 Heidi Health 合作，展示了使用 NVIDIA CUDA MPS 和 Triton 推理服务器在 Amazon EC2 上优化 ASR 推理成本的方法。通过 MPS 并发执行，GPU 利用率从 15-20% 提升，使 GPU 实例需求从 16 个减少到 4 个，成本降低 75%，同时保持亚秒级延迟。该方案结合 ONNX Runtime 与 TensorRT 优化编码器，并采用动态批处理和序列批处理策略。
正文节选
Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2 This post is a collaboration between AWS, NVIDIA and Heidi. Reducing automatic speech recognition (ASR) inference costs on Amazon Elastic Compute Cloud (Amazon EC2) becomes critical when GPU utilization per request is low but latency requirements are strict. A single ASR inference request typically uses only 15–20 percent of a GPU’s compute capacity, yet the default time-slicing behavior in NVIDIA CUDA® forces sequential access, lea