llama.cpp b11906:ggml-cpu 重写 concat 为单行 kernel
原标题:b11906
AI 摘要
llama.cpp 发布 b11906 版本,核心改动是在 ggml-cpu 中将 concat 操作重写为适用于所有数据类型的单行 kernel:每行目标数据来自 src0 或 src1,稠密行用一次 memcpy 完成(含量化类型),跨步源按编译期已知元素大小逐元素拷贝。同时改为按行跨线程切分并以 32 为块拷贝,优化转置源的访存局部性,并为 test_concat 增加转置 b 用例及性能测试。该版本同步提供 macOS、Linux、Windows、Android 等多平台多后端(CUDA、Vulkan、ROCm、SYCL、OpenVINO 等)预编译包。
正文节选
<details open> ggml-cpu : rewrite concat as one row kernel for all types (#31179) Every dst row comes from src0 or src1, or for dim 0 from the row of src0 followed by the row of src1, so a single kernel covers every type: a dense source row is one memcpy, quantized types included, and a strided source is copied element by element with the element size known at compile time. Rows are split across threads instead of i2 alone, which is often 1, and are copied in tiles of 32: for a transposed sou