NVIDIA 发布 AIPerf：大规模 LLM 推理基准测试工具
原标题：Benchmarking LLM Inference at Scale with AIPerf
AI 摘要
NVIDIA 发布 AIPerf，作为 GenAI-Perf 的继任者，从零重写以解决 LLM 推理基准测试中客户端成为瓶颈的问题。AIPerf 采用多进程架构，通过 ZMQ 协调工作进程与结果处理服务，支持 15+ 端点类型、多种公开数据集和可调负载模式。文章以 Qwen3-0.6B 配合 vLLM 为例，演示了安装、启动服务及运行 profile 的完整流程，并解释了 TTFT、ITL、请求延迟和输出吞吐量等核心指标。
正文节选
You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send curl commands, hand-roll an asyncio script, or vibe code yet another one-off load generator. All of these paths have the same problem: single-process performance limits, Python’s GIL capping concurrency, or numbers measured against a reference you built yourself. Either way, you end up with results you can’t fully trust, attached to tooling