llama.cpp b10956:SYCL 后端新增基数选择 TOP_K
原标题:b10956
AI 摘要
llama.cpp 发布 b10956 版本,为 SYCL 后端新增基于基数选择(radix select)的 TOP_K 实现,解决了此前 k>32 时被迫回退 CPU 的限制。新实现将直方图而非候选集放入 SLM,使内存占用与 k 无关,并支持在行数不足时把单行拆分到多个工作组以充分利用设备。实测在 ne=[131072,1] k=2048 时相对 CPU 回退路径加速 4.98 倍,端到端在 3x Arc Pro B60 上 tg64 从 5.91 提升至 6.05 t/s,困惑度无变化。
正文节选
<details open> sycl: rfc: Use radix select for top_k (#28670) * sycl: GPU-resident TOP_K for large k, parallelised over the device The SYCL backend refused GGML_OP_TOP_K above k = 32 and let it fall back to the CPU, a backend round-trip per call. The limit was not conservatism: the scan-merge kernels keep (split_block + 1) * k candidate (value, index) pairs in SLM, so at k = 128 a work-group already needs 132 KB and cannot launch. qwen4exp's sparse-attention indexer asks for k = 2048 in 12 la