返回全部动态

llama.cpp b11906:ggml-cpu 重写 concat 为单行 kernel

原标题:b11906

llama.cpp Releases一手来源开源质量 71

AI 摘要

llama.cpp 发布 b11906 版本,核心改动是在 ggml-cpu 中将 concat 操作重写为适用于所有数据类型的单行 kernel:每行目标数据来自 src0 或 src1,稠密行用一次 memcpy 完成(含量化类型),跨步源按编译期已知元素大小逐元素拷贝。同时改为按行跨线程切分并以 32 为块拷贝,优化转置源的访存局部性,并为 test_concat 增加转置 b 用例及性能测试。该版本同步提供 macOS、Linux、Windows、Android 等多平台多后端(CUDA、Vulkan、ROCm、SYCL、OpenVINO 等)预编译包。

以上摘要由 AI 生成,可能存在误差。事实请以原文为准。

正文节选

<details open> ggml-cpu : rewrite concat as one row kernel for all types (#31179) Every dst row comes from src0 or src1, or for dim 0 from the row of src0 followed by the row of src1, so a single kernel covers every type: a dense source row is one memcpy, quantized types included, and a strided source is copied element by element with the element size known at compile time. Rows are split across threads instead of i2 alone, which is often 1, and are copied in tiles of 32: for a transposed sou


发布时间:2026-10-27 03:45
抓取时间:2026-10-27 04:18
来源机构:ggml-org
阅读原文github.com