相同输出不同金标：参考选择如何改变多语言基准分数
原标题：Same Output, Different Gold: Measuring How Reference Choice Moves a Multilingual Benchmark Score
AI 摘要
该研究测量了多语言PII抽取基准中参考标准（gold）选择对评分的影响。作者固定系统输出、替换参考标准，发现分数变化可达数个百分点，且两个系统间的比较结果也会改变，Hindi语上的差距甚至变号。原因是聚合默认规则在未触发裁决时通常保留标注者A，使发布的参考标准部分复制了被评分的输出。作者提出用参考敏感性区间来量化这一评估不确定性。
正文节选
TAE (Trust-AI-Eval): Can We Trust AI Evaluation? Same Output, Different Gold: Measuring How Reference Choice Moves a Multilingual Benchmark Score Abstract A benchmark score compares a system output against a reference, and methodological attention falls almost entirely on the first term. We measure the second. The retained annotation record of a six-language benchmark for personally identifiable information contains two independent annotator labellings, the aggregate shipped as gold, and a revie