llama.cpp b11707:Granite switch 支持 256 选项标记
原标题:b11707
AI 摘要
llama.cpp 发布 b11707 版本,主要合并了 PR #30693,为 Granite switch decision 模型支持最多 256 个单 token 语义选项标记。转换阶段会遍历具体模型的词表,发现 1 个及以上纯字母的单 token 序列(如 A、AA、BB),以突破原先大小写字母加数字仅 66 个标记的限制,并让 C++ 侧计算出的标签成为模板的权威来源。该改动提升了 Granite 模型在 llama.cpp 上的选项判定能力,同时发布覆盖 macOS、Linux、Windows、Android 的多后端预编译包。
正文节选
<details open> model : Granite switch decision 256 options (#30693) * feat(convert): Attempt to come up with 256 single-token, semantic option markers Each switch model needs to identify a collection of single-token option markers that can be mapped to their respective logits for the output probability distribution. The simple upper/lower/numbers set only yields 66 unique markers, but the model can support up to 255. The challenge is that which combinations of characters result in single-toke