llama.cpp b10985:RPC 哈希缓存仅限权重张量
原标题:b10985
AI 摘要
llama.cpp 发布 b10985 版本,修复了 RPC 后端的哈希缓存问题。此前 ggml_backend_rpc 会对所有超过 HASH_THRESHOLD 的传输(包括调度器在节点间复制的激活值)进行哈希并写入 rpc-server 的磁盘缓存,导致 Qwen3.8-Flash-Next 双节点拆分下一天内缓存膨胀至 1.4 TB。补丁改为仅对标记为 GGML_BACKEND_BUFFER_USAGE_WEIGHTS 的权重张量走哈希缓存路径,并在 SET_TENSOR 消息中新增 cache_flag 字节、提升 RPC_PROTO_MAJOR_VERSION。
正文节选
<details open> rpc : hash-cache only weights (#28789) * rpc : hash-cache only weights ggml_backend_rpc_buffer_set_tensor and ggml_backend_rpc_set_tensor_async hashed every transfer above HASH_THRESHOLD and let `rpc-server -c` serve it from its file cache. The cache is meant for weights, but the activations ggml_backend_sched copies between backends took the same path: with a two-node split of Qwen3.8-Flash-Next every prefill ubatch above 10 MB was hashed, written to the worker's cache directo