返回全部动态
llama.cpp b10840 发布：CUDA 量化加速与 DGX Spark 预取优化
原标题：b10840
AI 摘要
llama.cpp 发布 b10840 版本，主要更新包括在 CUDA 后端为 Q4_K/Q5_K 量化模型引入无分支解包以加速 mmvq 操作，并针对 DGX Spark 添加 L2 预取优化。该改动在批量大小大于 1 时提升性能，并调整了切换点以适应更多模型。同时提供了多平台二进制下载。
以上摘要由 AI 生成，可能存在误差。事实请以原文为准。
正文节选
<details open> CUDA: branchless Q4_K/Q5_K unpack to speed up mmvq, L2 prefetch on DGX Spark (#26705) * Update Q4_K and Q5_K to use branchless computation, which stops the scale unpack being re-executed for every column in mmvq, improving perf at batch sizes > 1 * Gating the change off from DGX Spark due to no gain * Adding prefetch gated to Spark, making branchless change in Q4_K and Q5_K general and modifying switch points based on latest perf data * Guard the mmvq L2 prefetch against MUSA
发布时间：2026-09-07 22:31
抓取时间：2026-09-07 22:55
来源机构：ggml-org