返回全部动态
llama.cpp b11567:新增推理计算图重分配回调报告
原标题:b11567
AI 摘要
llama.cpp 发布 b11567 版本,新增可选的 ggml_backend_sched_realloc_callback 回调机制,允许用户代码在推理过程中图缓冲区不足需重新分配时收到通知。llama_context 在创建调度器时注册该回调,将后端分配变化视为预期情况,其余按图大小变化计数,并在性能统计中输出这两个计数器。该版本同时提供 macOS、Linux、Windows、Android 等多平台预编译包。
以上摘要由 AI 生成,可能存在误差。事实请以原文为准。
正文节选
<details open> llama : report compute graph reallocations during inference (#30329) Add an optional ggml_backend_sched_realloc_callback that user code registers with ggml_backend_sched_set_realloc_callback to be notified when the buffers reserved for a graph are not sufficient and have to be reserved again. The callback reports whether the backend assignments and the graph size changed since the previous scheduling. llama_context registers the callback when creating the scheduler (wrapped int
发布时间:2026-10-12 20:26
抓取时间:2026-10-12 21:27
来源机构:ggml-org