BestRand：稳定低成本基准压缩方法
原标题：Paper page - Random Is Almost Enough? Stable and Low-Cost Benchmark Compression via BestRand
AI 摘要
该论文研究基准压缩（benchmark compression）问题，即用小子集近似全基准评测结果以降低评测成本。作者推导出子集估计误差分布的解析近似，发现误差呈长尾分布：多数子集误差小，少数子集误差极大，因此纯随机采样虽大概率表现良好但不稳定，且这种不稳定性在现代基础模型中固有存在。为此提出 BestRand 方法，在保留随机采样效率的同时显著削减高误差尾部，在 70+ 文本、多模态与 agent 基准及 VLA 评测上验证，高误差风险降低 50.2%、估计误差平均降低 16.4%，构建成本仅为对比方法的 51.3%，OOD 估计误差平均降低 21.8%，代码与数据已开源。
正文节选
Random Is Almost Enough? Stable and Low-Cost Benchmark Compression via BestRand Abstract Benchmark compression reduces the often prohibitive time and cost of model evaluation by evaluating models on a small benchmark subset that closely matches full-benchmark results. To quantify estimation error, we derive an accurate analytical approximation to the error distribution when estimating full-benchmark scores from a subset, showing that most subsets yield small errors, while a few produce much larg