基准即难度上的经验测度：诊断失配与覆盖缺口
原标题：Paper page - Your Benchmark Is an Empirical Measure Over Difficulty
AI 摘要
该论文将基准评测形式化为难度上的经验测度，识别出测度失配与覆盖缺口两种核心失效模式，并提出两个带理论保证的诊断指标。作者对覆盖7个领域的25个流行基准进行实证分析，发现许多基准隐式偏重某些难度区间，导致排行榜结论需谨慎解读，另一些则存在明显覆盖缺口。论文进一步提出BenchFill，一种面向难度的题目改写算法，用于在覆盖不足的难度区域生成新样例以缩小局部缺口。
正文节选
Your Benchmark Is an Empirical Measure Over Difficulty Abstract This work formalizes benchmark evaluation through difficulty measures, identifies failure modes of measure mismatch and coverage gaps, and proposes diagnostic metrics and a rewriting algorithm to improve benchmark coverage. A benchmark is not merely a collection of test questions; it naturally induces an empirical measure over difficulty. In this work, we develop this measure-based view of benchmark evaluation. This perspective allo