返回全部动态

llama.cpp 新增决策模型与 /v1/systemone 端点

原标题:New in llama.cpp: Decision Models

Hugging Face Blog一手来源产品发布质量 80

AI 摘要

llama.cpp 新增 /v1/systemone 端点,支持决策模型(Decision Models)。这类模型不生成文本,而是对给定选项打分并返回概率,单次前向传播即可完成,适用于请求路由、内容审核、Agent 步骤校验等场景。官方同时列出 Julia-1、Laya、Kev-4B、lev、OpenJev 等开源模型,其中 OpenJev 支持图像输入,API 沿用 TypeSafe Jev 模型的 System One 格式,现有客户端只需更换 base URL。

以上摘要由 AI 生成,可能存在误差。事实请以原文为准。

正文节选

/v1/systemone endpoint. You send a state (text, JSON, a screenshot) and typed questions. The model returns a probability for each option in a single forward pass. The API follows the System One format introduced with TypeSafe's Jev model, so existing clients only need a new base URL. Implementation details are in PR #29818. What is a decision model? A model that answers by scoring the options you give it, instead of generating text. A chat model spends one forward pass per output token, and its


发布时间:—
抓取时间:2026-10-03 01:13
来源机构:Hugging Face
阅读原文huggingface.co