llama.cpp b11513:CUDA top-k 算法优化
原标题:b11513
AI 摘要
llama.cpp 发布 b11513 版本,主要改进 CUDA 后端的 top-k 算法选择。新实现用基于行的 radix select 替代 CUB 的逐行 DeviceTopKKernel,在 qwen4exp 34,816 tokens 场景下将 top-k 从 1,671,253 次启动/5,761.8 ms 降至 2,329 次/941.8 ms。同时按形状选择实现(短行用 bitonic、多长行用 radix select、单长行用 DeviceTopK 或 CUB argsort),并新增性能测试用例。
正文节选
<details open> CUDA: improve top-k algorithm selection (#28713) * CUDA: radix top-k for large row counts Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select, gated on GGML_CUDA_TOPK_RADIX_MIN_ROWS. On qwen4exp at 34,816 tokens this cuts top-k from 1,671,253 launches / 5,761.8 ms to 2,329 / 941.8 ms. * CUDA: select the TOP_K implementation by shape Replace the nrows/ncols special case with the decision boundary from #28547 (as implemented in #29278): bitonic for short