llama.cpp b12191：MoE 缓存支持 CPU 分担未命中专家

AI 摘要

llama.cpp 发布 b12191 版本，为 MoE 缓存引入 CPU 分担机制：通过 --moe-cache-cpu-share 参数，将部分未命中专家在 CPU 上计算而非上传至设备，并新增 moe-cache-tune.py 调优脚本自动推荐缓存大小与 CPU 分担比例。该改动还调整了调度器与图构建逻辑，使 CPU 部分可异步执行、不阻塞设备计算，旨在降低 MoE 推理的显存占用与传输开销。

正文节选

<details open> llama_moe_cache: run missed experts on CPU, add tuning script (#30966) * moe-cache : compute part of the misses on the CPU --moe-cache-cpu-share F computes a share F of the MoE cache misses on the CPU from the host experts instead of uploading them. The graph builds the expert FFN twice: on the device from the cache slots, where the misses left to the CPU read zero slots, and on the CPU from the host experts, where the experts computed on the device have id -1. The two results

发布时间：2026-11-08 05:11

抓取时间：2026-11-08 05:45

来源机构：ggml-org