RAGFlip：衡量检索器升级中的查询级负翻转
原标题：RAGFlip: Measuring Query-Level Negative Flips in Retriever Upgrades
AI 摘要
该论文提出 RAGFlip 方法，用于衡量检索器升级时在查询级别出现的负翻转（negative flips），即 BM25 原本能检索到相关段落的查询在替换检索器后失败的情况。研究在 Natural Questions、HotpotQA 和 FiQA 三个 BEIR 数据集上评估 BGE-large、E5-large-v2 和 SPLADE，发现所有替换检索器虽提升整体覆盖率，但在各设置下均出现负翻转，BM25 成功查询的丢失率在特定深度下达 8.6–37.5%。作者建议在评估检索器更新时，除聚合指标外还应考虑查询级兼容性。
正文节选
RAGFlip: Measuring Query-Level Negative Flips in Retriever Upgrades Abstract Retriever upgrades are typically evaluated using aggregate metrics, which can hide regressions on queries the previous retriever already served correctly. We study these regressions as negative flips: queries for which BM25 retrieves a judged relevant passage and the replacement does not. We evaluate BGE-large, E5-large-v2, and SPLADE on three BEIR collections Natural Questions, HotpotQA, and FiQA across five retrieval