TRACES基准：评估大语言模型科学推理的认知可靠性

AI 摘要

TRACES 基准测试通过 42 篇撤稿、欺诈或伪科学论文，评估大语言模型在科学推理中的认知可靠性。测试发现，30 个模型在 71% 以上的智能体探针中失败，22 个模型失败率超过 90%，表明模型拒绝不可靠前提的能力主要基于主题安全行为而非真正的认知能力。该研究呼吁为科学部署的语言模型建立防护基础设施。

正文节选

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs Abstract Large language models are being proposed as agents in scientific workflows, in domains where no downstream verifier exists. Such deployment assumes the model can distinguish reliable scientific literature from unreliable literature, a capability that has not yet been directly measured. Existing benchmarks evaluate factuality on questions with known answers; the failure mode we target here is different. We intr

发布时间：2026-08-13 12:00

抓取时间：2026-08-13 12:02

来源机构：arXiv