llama.cpp b11597：池化嵌入跳过 LM head 优化

AI 摘要

llama.cpp 发布 b11597 版本，主要优化了池化嵌入（pooled embeddings）场景下的计算图构建。此前即使上下文只提取池化嵌入，模型仍会展开 LM head 并计算每个 token 的 n_vocab 个 logits，造成冗余计算和主机内存占用。本次改动让模型仅声明 t_logits，并在池化类型非 NONE 时跳过 LM head 与采样层，同时新增 build_logits() 并调整调用顺序。

正文节选

<details open> graph: skip the LM head for pooled embeddings (#30445) * graph: skip the LM head for pooled embeddings A context that extracts pooled embeddings outputs one vector per sequence, but every model still expanded its LM head, so the graph computed n_vocab logits per token and the context kept a host buffer for them. The models now only declare t_logits, and set_outputs() expands it unless the context extracts embeddings with a pooling type other than NONE. Since the toggle changes

发布时间：2026-10-14 04:16

抓取时间：2026-10-14 05:11

来源机构：ggml-org