EvalDetectBench：测量前沿语言模型评估意识的新基准
原标题：EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models
AI 摘要
EvalDetectBench是一个用于测量前沿大语言模型评估意识的开源基准和流水线，兼容任何Inspect兼容的评估。该基准发现现有文献中的两个方法论选择会引入系统性偏差：生成部署转录的模型身份占测量方差的11.25%并可重新排序模型排名，且为某一模型选择的高性能提示在其他模型上可能接近随机水平。EvalDetectBench通过逐模型探针校准和分层生成器协调程序纠正了这两个问题，并发布了开源流水线和数据集。
正文节选
EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models Abstract Frontier large language models can often recognize when they are being evaluated, a capability known as evaluation awareness. If models behave differently in evaluations than in deployment, this undermines the validity of evaluation results, which are a crucial component of current AI safety frameworks. We introduce EvalDetectBench, an open pipeline and benchmark for measuring evaluation awarenes