llama.cpp 为 SSM_SCAN 内核新增 d_state=64 快速路径
原标题:b12207
AI 摘要
llama.cpp 发布 b12207 版本,在 ggml-cuda 的 SSM_SCAN 内核中新增 d_state=64 的快速路径,以支持 Zamba2 等使用状态大小 64 的循环扫描架构。此前这类模型的扫描步骤全部回退到 CPU,生成速度仅 6-7 tok/s;新增模板实例化后,Zamba2-7B-Instruct 实测吞吐提升约 35%(11 提升至 15.1 tok/s),并通过 3 个基于 Zamba2 真实超参数的新测试用例验证正确性。不过仍有约 28% 的扫描调用因未诊断原因回退到 CPU,尚未完全达到与其他架构持平的性能。
正文节选
<details open> ggml-cuda : add d_state=64 fast path to SSM_SCAN kernel (#31818) Zamba2 and similar architectures use a state size of 64 in their recurrent-scan step, which fell outside the GPU kernel's existing fast paths (96, 128, 256). Every scan step was falling back to CPU, bottlenecking generation to 6-7 tok/s against the 50-70 tok/s band other architectures achieve on the same hardware. Add a 4th template instantiation for d_state=64, following the existing pattern; no change to the und