新开源基准揭示推理是AI云运维的最大瓶颈
原标题：New open-source benchmark reveals reasoning as the biggest bottleneck for AI in cloud operations
AI 摘要
密歇根大学Ang Chen团队发布开源基准CloudOpsBench 1.0，包含45个AWS云上DevOps任务，用于评估AI智能体处理修复服务、删除资源、限制权限等典型运维任务的能力。测试中表现最好的Claude Fable 5.1仅解决60%的任务，GPT-6 Astra解决53%。团队分析发现约80%的失败源于推理问题（误判任务范围、未检查所有相关服务、过早停止），约20%为错误操作，部分对基础设施造成不可逆后果。
正文节选
New open-source benchmark reveals reasoning as the biggest bottleneck for AI in cloud operations CloudOpsBench is a new open-source benchmark that measures how well AI agents handle typical DevOps tasks in the cloud, from fixing services and removing resources to restricting access rights, rolling out updates, and decommissioning projects. Version 1.0 includes 45 tasks on AWS and was built by experienced DevOps engineers. Even the best models fail at many of the tasks. Claude Fable 5.1 solved 60