基准污染缓解新指标与策略：SA-PPG 与 RailCap
原标题：Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination
AI 摘要
Hugging Face 每日论文发布了一篇关于基准测试污染缓解的研究。论文指出现有评估指标 G-AP 存在缺陷，并提出新指标 SA-PPG 和缓解策略 RailCap。实验表明，SA-PPG 揭示先前策略的恢复效果被高估，而 RailCap 实现了最低的 SA-PPG。
正文节选
Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination Abstract Test data from public benchmarks inevitably leaks into pretraining corpora, inflating evaluation scores once memorized. Contamination mitigation evaluation intervenes in the decoding process to suppress memorization and restore a contaminated model's genuine capability, but its prevailing metric, the G-AP (Gap of Aggregate Performance), is flawed. Discrete corr