智能体数据科学中的沉默失败
原标题：The Silent Failures in Agentic Data Science
AI 摘要
该论文指出，当前对数据科学智能体（agentic DS）的评估仅关注下游预测性能和代码正确性，忽略了数据科学方法论和领域知识，从而掩盖了数据泄漏、验证不当等“沉默失败”。作者从三个角度评估：对220个数据科学任务的大规模失败模式分析、可复用的受控测试套件、以及与Kaggle专家方案的定性对比。结果发现大多数智能体生成的流程至少存在一种失败模式，且数据泄漏失败率与虚高的预测性能正相关，智能体在问题定义和特征工程上弱于人类专家。
正文节选
leakbadge \tl_set:Ne\otherbadgeotherbadge \tl_set:Ne\tfbadgetfbadge The Silent Failures in Agentic Data Science Abstract Large language models have significantly lowered the barrier to writing code. Naturally, this has also led to the development of data science agents that promise automating the design and execution of data science from task formulation to model development. Existing evaluations of agentic data science (agentic DS) assess downstream predictive performance and code correctness,