llama.cpp b12065:修复 kv-cache 零 kv_rows 问题
原标题:b12065
AI 摘要
llama.cpp 发布 b12065 版本,修复了 kv-cache 的一个缺陷:当某个缓存共享另一个缓存的 cell(例如 gemma4 assistant 场景)且其序列尚未分配任何 cell 时(如服务器预热阶段),get_n_kv_rows() 会返回 0,导致计算图回退到稠密 KQ mask 并触发调度器重新规划;后续使用更大 mask 的计算图在 GGML_SCHED_DEBUG_REALLOC=1 下会中止。该版本同时提供 macOS、Linux、Windows、Android 等多平台及 CUDA、Vulkan、ROCm、SYCL、OpenVINO 等后端的预编译包。
正文节选
<details open> kv-cache : never return 0 kv_rows for sequences without cells (#31570) A cache that shares the cells of another one (f.ex. the gemma4 assistant) can build a graph before its sequences have any cells, f.ex. at the server warmup. get_n_kv_rows() then returned 0, which means that kv_rows are not used, so the graph fell back to the dense KQ mask and the scheduler re-planned for it. Later graphs with larger masks then aborted with GGML_SCHED_DEBUG_REALLOC=1. </details> **Website:**