为语言模型智能体保障组织轨迹证据:片段、方法与残差
原标题:Organising Trajectory Evidence for Language-Model Agent Assurance: Fragments, Methods, and the Residual
AI 摘要
该论文提出一种面向语言模型智能体保障的统一框架,用双层逻辑(外层有限轨迹时序逻辑、内层关于论证结构成立性的逻辑)与信息片段、保障账本组织来自规则检查器、支持检查器、前缀监控、执行门和稀有事件估计器等不同方法的证据。在456条电信轨迹上,基准预言机标记231条,加入规则检查器、支持替代和前缀监控基线后并集分别升至391、401、407条,各方法贡献了其他方法遗漏的标记,执行门使某条禁令在受控部署中变为精确。未被标记的49条轨迹及未满足或未处理的义务构成残差,需通过发现新需求并引入更强检查器来缩减。
正文节选
Public preprint Organising Trajectory Evidence for Language-Model Agent Assurance: Fragments, Methods, and the Residual Abstract. Methods for assessing language-model agents include rule checkers over logs, analyses of skill coverage and composition, support checkers, prefix monitors, execution gates, and rare-event estimators. Each observes a different part of a run and makes a claim of a different strength, and no common account says how these claims combine or what they leave unchecked. We gi