AIFaultBench:AI 软件真实缺陷可复现基准
原标题:AIFaultBench: A Reproducible Benchmark of Real-World Faults in AI Software Systems
AI 摘要
研究者发布 AIFaultBench,一个包含来自 76 个组织的 105 个 AI 软件仓库、共 770 个真实缺陷的可复现基准,覆盖机器学习、深度学习、强化学习、大语言模型、智能体 AI 及 AI 工具与基础设施六类系统。每条缺陷记录关联原始报告、源码版本、环境配置脚本、复现触发条件和调查证据,其中 652 个(84.7%)在记录条件下成功复现,其余 118 个也记录了未复现原因。该基准旨在支持 AI 故障实证研究、调试工具评估、实际故障排查和教学。
正文节选
AIFaultBench: A Reproducible Benchmark of Real-World Faults in AI Software Systems Abstract AI-based software systems remain difficult to debug because their failures can involve interactions among code, models, data, dependencies, hardware, and execution environments. Existing benchmarks cover specific classes of AI systems, while concerns about fault realism and reproducibility highlight the need for documented execution conditions and evidence. We present AIFaultBench, a benchmark of 770 real