数绷带:论如何衡量 AI 节省的时间
原标题:Counting the Bandages: On measuring the time that AI saves
AI 摘要
该论文提出用「账本视角」而非「秒表视角」衡量 AI 节省的时间,将时间节省与逃逸缺陷(excess escaped defects)结合,并给出决策指标「盈亏平衡缺陷成本」。作者复现 METR 开发者研究,发现同一批 246 个问题因估计量不同可得出 19% 或 24% 的减速结论;并指出在 GDPval 的逐项审查流程中,仅审查时间就限制了 GPT-5 的加速幅度。小型预注册实验显示,AI 审查者被告知起草者真实错误率时会更多检查,但呈全有或全无的切换,被低估错误率时检查更少、放行错误更多。
正文节选
[Extension=.otf,UprightFont=*-regular,BoldFont=*-bold,ItalicFont=*-italic,BoldItalicFont=*-bolditalic] \setmathfonttexgyrepagella-math.otf [Scale=0.95] Counting the Bandages On measuring the time that AI saves Abstract I consider what it means for AI assistance to save time. I contrast a stopwatch view, on which time saved is whatever the clock records, with a ledger view, on which it is a counterfactual that has to be read next to the errors the work lets through, and I argue for the latter. Th