返回全部动态
Cloudflare 优化 Kimi 与 GLM 推理：KV 缓存量化与权重压缩
原标题：Smaller, faster, safer: running Kimi and GLM at scale
AI 摘要
Cloudflare 在 Workers AI 上通过 KV 缓存量化（FP8）和模型权重压缩（INT4）优化了 Kimi 和 GLM 等大型 MoE 模型的推理效率，使内存占用减半、吞吐量提升，且模型精度几乎无损。这些技术结合预填充/解码分离架构，支持更多并发请求并降低成本，相关优化已通过 SGLang 框架开源。
以上摘要由 AI 生成，可能存在误差。事实请以原文为准。
正文节选
Workers AI runs inference for some of the best open models in the world on GPUs in Cloudflare data centers close to your users. Two of the most capable, and most demanding, are Moonshot's Kimi K-series and Z.ai's GLM. They are large, long-context, mixture-of-experts models, and they are wonderful to use. They are also very hard to serve efficiently because of memory constraints. We've written before about how we serve large models on Workers AI and about separating the prefill and decode phases
发布时间：2026-08-03 21:00
抓取时间：2026-08-03 21:41
来源机构：Cloudflare