构建网络安全评估的模式

AI 摘要

Eugene Yan 撰文探讨了构建网络安全评估（cybersecurity evals）的模式，介绍了评估模型漏洞利用能力的通用框架，包括沙箱目标、难度输入、工具和评分器。文章重点分析了 Cybench 等基准测试的设计与结果，指出当前模型在复杂任务上表现有限，如 Claude 3.5 Sonnet 在无引导模式下成功率仅 17.5%。

正文节选

How do we evaluate if a model can find and exploit security vulnerabilities? How do we know when agents become useful for defenders, and when they cross the threshold into uplifting attackers? Here, we discuss some benchmarks that measure this, from capture-the-flag exercises to data exfiltration on a 50-host network. Before diving into the benchmarks, I think it helps to understand the common pattern they share, largely based on four primitives. (You’ll notice that it’s similar to general evals

发布时间：2026-06-21 08:00

抓取时间：2026-08-02 00:25

来源机构：Eugene Yan