因果发现基准:LLM 能否自主开展有效实验?
原标题:Causal Discovery Bench: Can LLMs Autonomously Conduct Effective Experiments?
AI 摘要
该论文提出 Causal-Discovery-Bench,一个可验证沙盒,用可配置的结构方程模型和资源约束评估 LLM 智能体自主设计实验、进行因果发现的能力。研究覆盖 5 家厂商的 13 个模型、3802 个回合,发现智能体 94.5% 的错误源于实验设计而非因果推断,且单纯放宽资源约束不能可靠解决顺序发现失败。基础统计工具能显著提升多个模型家族的表现,而高级工具在模型缺乏底层能力时收益有限。
正文节选
Causal Discovery Bench: Can LLMs Autonomously Conduct Effective Experiments? Abstract As large language model (LLM) agents get actively adopted for autonomous research, they must not only reason over existing evidence, but also design informative experiments, allocate resources, analyze sequential observations, and decide when to stop and make conclusions. Yet these capabilities remain poorly measured. We introduce Causal-Discovery-Bench (https://github.com/mshen1019/causal-discovery-bench), a v