检测不够：将推理轨迹作为可测试的程序假设

AI 摘要

该论文研究在无参考解、无评判模型、无权重更新的条件下，如何验证大语言模型单条推理轨迹的可信度。作者在冻结的 LLaMA-2-7B-chat 上基于 GSM8K 测试发现，静态自检的 AUROC 仅达 0.741，因为 67.1% 的错误轨迹并未包含错误等式；他们提出将轨迹抽象为程序并扰动题目数量的变形探针，71.0% 的测试题可构造有效扰动，但该探针也会误驳 45.5% 的正确轨迹。结论是检测并非瓶颈，修复才是开放问题：在 12.3% 的修复率下，即使完美检测器在相同重试预算下也只能多恢复约 66/1319 道题。

正文节选

Abstract When a language model is deployed, no reference solution exists, so the reasoning trace it writes is the only evidence for judging its answer. The reference-free substitutes each fail differently: self-assessment degrades the answer it was meant to protect, sampling agreement measures stability rather than correctness, and perturbation studies read the expected answer off a human-written template, grading a model rather than a trace. We ask how far a single trace can be verified with no

发布时间：2026-10-23 12:00

抓取时间：2026-10-23 13:49

来源机构：arXiv