返回全部动态

llama.cpp 为 SpacemiT X60 新增 Q8_0 IME1 矩阵内核

原标题:b11408

llama.cpp Releases一手来源开源质量 78

AI 摘要

llama.cpp 发布 b11408 版本,为 ggml-cpu 后端新增了针对 SpacemiT X60 的 Q8_0 IME1 矩阵内核。此前 X60 上 IME 加速仅覆盖 Q4_0/Q4_1/Q4_K,Q8_0 因无 IME1 内核且未编译 repack 路径而无加速,预填充速度约为 Q4_0 的十分之一。新增 make_block_q8_0x16、ime1::gemm_kernel_i8i8 及 4 行激活量化后,在 Milk-V Jupiter 上 Qwen2.5-0.5B-Instruct Q8_0 的 pp128 从 10.70 t/s 提升至 93.87 t/s。

以上摘要由 AI 生成,可能存在误差。事实请以原文为准。

正文节选

<details open> ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 (#28479) * ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 On the SpacemiT X60, IME matrix acceleration only covered Q4_0/Q4_1/Q4_K. Q8_0 had no IME1 kernel, and since the SpacemiT build sets GGML_CPU_REPACK=OFF there was no repack path compiled in either, so Q8_0 had no accelerated path at all and ran roughly ten times slower than Q4_0 for prefill on the same board. - add make_block_q8_0x16 and the Q8_0 repack en


发布时间:2026-10-05 16:35
抓取时间:2026-10-05 16:55
来源机构:ggml-org
阅读原文github.com