返回全部动态

llama.cpp b10985:RPC 哈希缓存仅限权重张量

原标题:b10985

llama.cpp Releases一手来源开源质量 72

AI 摘要

llama.cpp 发布 b10985 版本,修复了 RPC 后端的哈希缓存问题。此前 ggml_backend_rpc 会对所有超过 HASH_THRESHOLD 的传输(包括调度器在节点间复制的激活值)进行哈希并写入 rpc-server 的磁盘缓存,导致 Qwen3.8-Flash-Next 双节点拆分下一天内缓存膨胀至 1.4 TB。补丁改为仅对标记为 GGML_BACKEND_BUFFER_USAGE_WEIGHTS 的权重张量走哈希缓存路径,并在 SET_TENSOR 消息中新增 cache_flag 字节、提升 RPC_PROTO_MAJOR_VERSION。

以上摘要由 AI 生成,可能存在误差。事实请以原文为准。

正文节选

<details open> rpc : hash-cache only weights (#28789) * rpc : hash-cache only weights ggml_backend_rpc_buffer_set_tensor and ggml_backend_rpc_set_tensor_async hashed every transfer above HASH_THRESHOLD and let `rpc-server -c` serve it from its file cache. The cache is meant for weights, but the activations ggml_backend_sched copies between backends took the same path: with a two-node split of Qwen3.8-Flash-Next every prefill ubatch above 10 MB was hashed, written to the worker's cache directo


发布时间:2026-09-15 22:19
抓取时间:2026-09-15 23:26
来源机构:ggml-org
阅读原文github.com