Cultivar：用于检测污染和本地化鲁棒性的对比翻译基准
原标题：Cultivar: A Contrastive and Locale-Oriented Translation Benchmark for Investigating Contamination and Localisation Robustness
AI 摘要
研究人员提出了一种源对比评估方法，通过本地化基准Cultivar来检测多语言翻译模型中的数据污染并评估其本地化鲁棒性。Cultivar是FLORES的本地化子集，与未本地化的对应部分配对使用时，性能差异可揭示污染和鲁棒性问题。对32个开源权重模型的基准测试显示，MT专用模型鲁棒性较差，部分模型可能过拟合FLORES，且模型对US内容的翻译普遍优于其他地区。
正文节选
Cultivar: A Contrastive and Locale-Oriented Translation Benchmark for Investigating Contamination and Localisation Robustness Abstract Researchers propose source-contrastive evaluation via a localized benchmark to detect data contamination and assess localization robustness in multilingual translation models. Multilingual translation benchmarks are typically sourced in English and translated into other languages, treating language pairs as the unit of evaluation---a design that is prone to conta