评估长上下文问答系统：忠实性与有用性

AI 摘要

Eugene Yan 撰文探讨长上下文问答系统的评估方法，提出忠实性和有用性两个核心维度，并讨论了评估数据集构建、人类标注与 LLM 评估器，以及多个基准测试。文章强调忠实性要求答案严格基于文档，有用性则需相关、全面且简洁，并指出两者间的张力。最后给出针对具体用例的评估建议。

正文节选

While evaluating Q&A systems is straightforward with short paragraphs, complexity increases as documents grow larger. For example, technical documentation, novels and movies, as well as multi-document scenarios. Although some of these evaluation challenges also appear in shorter contexts, long-context evaluation amplifies issues such as: In this write-up, we’ll explore key evaluation metrics, how to build evaluation datasets, and methods to assess Q&A performance through human annotations and LL

发布时间：2025-06-22 08:00

抓取时间：2026-08-02 00:25

来源机构：Eugene Yan