llama.cpp b11182:新增精度策略与 W4A4 推理路径
原标题:b11182
AI 摘要
llama.cpp 发布 b11182 版本,核心变更是新增 llama_prec_policy 精度策略并引入模型驱动的 W4A4(4位权重/4位激活)推理路径(PR #24364),由 NVIDIA 工程师 ynankani 提交,Georgi Gerganov 等人参与评审。该改动涉及 ggml CUDA 后端 mmq.cu 的 MXFP4 调度调整,并修复了 HIP 平台 launch_bounds 重载导致的 CI 失败。同时发布了覆盖 macOS、Linux、Windows、Android 的多平台预编译二进制包。
正文节选
<details open> llama : add `llama_prec_policy` + model-driven W4A4 path (#24364) * Rebase and update based on #26675 Signed-off-by: ynankani <ynankani@nvidia.com> * CI failure fix(launh_bounds overload on HIP) and cleanup Signed-off-by: ynankani <ynankani@nvidia.com> * Address review comments Signed-off-by: ynankani <ynankani@nvidia.com> * Use ggml tensor instead of name in act policy map Signed-off-by: ynankani <ynankani@nvidia.com> * Address review comments and cleanup Signed-off-by