DRAFT-Bench:评估前沿智能体的专业设计全生命周期能力
原标题:DRAFT-Bench: Evaluating Frontier Agents Across the Professional Design Lifecycle
AI 摘要
研究者提出 DRAFT-Bench,一个面向专业设计全生命周期的智能体评测基准,覆盖需求获取、创意探索、受限编辑、迭代修改与生产交付等阶段,并引入 Requirement-to-Artifact 框架,把不断变化的需求、受保护属性、产物版本与交付义务关联起来。作者在 35 份设计简报、43 种条件下对比 Codex/Astra 与 Claude Code,另设 40 个场景的 RFQ 赛道;Astra 完成 43/43 个设计条件,Claude 完成 35/43 个。所有计分核心运行都获得结构分,但语义检查在 Astra 的 11 个、Claude 的 5 个核心条件中标记出违规;这些是模型判断,并非专家确认的失败率。研究强调应区分执行完成、需求合规与专业验收,且因赛道未配对、工具不同,不能据此得出 RFQ 到交付的成功率或纯模型排名。
正文节选
DRAFT-Bench: Evaluating Frontier Agents Across the Professional Design Lifecycle Abstract Can frontier general-purpose agents understand client intent, explore creative directions, execute revisions, and prepare production deliverables? We introduce DRAFT-Bench (Design Reliability Across Feedback and Tasks), a lifecycle-oriented evaluation of requirement elicitation, creative exploration, constrained editing, iterative revision, and production handoff. Its Requirement-to-Artifact framework links