DeepSeek 开源生产推理栈 DeepInfer，称单集群日处理 155 万亿 token
原标题：[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
AI 摘要
DeepSeek 在 PyTorch Conf 首日开源了其生产推理栈 DeepInfer，该引擎基于 vLLM 定制，支持分布式推理、投机解码与结构化输出，并可在 NVIDIA 与华为昇腾平台运行。官方披露单集群日处理 155 万亿 token、96% 输入缓存命中、约 80% 推理毛利率与 81 天回本周期，但 KV 缓存管理与负载均衡等核心组件尚未完全开源。vLLM 团队回应称在 GB200 匹配配置下解码吞吐与 DeepInfer 公布结果差距在 6% 以内，引发关于其效率独特性和推理盈利能力的讨论。
正文节选
Proudly brought to you by SerpAPI: Today’s release covers a LOT of ground, but you can scrape all this into markdown for your agents! 50% token cost of JSON. We last covered the Inference Inflection in April, and historically most of the advances have been closed source decacorns from Fireworks to Baseten to Modal. Well, time for the open source response. It is day 1 of PyTorch Conf, and DeepSeek timed an intercontinental ballistic missile into the discussion by open sourcing DeepInfer, their pr