压缩的代价：事实幻觉的率失真极限

AI 摘要

该论文提出一个信息论框架，将闭卷问答中的事实幻觉分解为两个可分离的来源：未观测事实的覆盖缺失，以及已观测事实因有限记忆被迫压缩存储而导致的失真。作者用覆盖-压缩模型证明了一个率失真下界，并通过理论模拟和在现代语言模型上的受控事实注入实验验证了预测特征。该工作为选择性记忆、强制压缩、检索和长上下文组织等行为提供了统一的信息论解释。

正文节选

The Cost of Compression: A Rate-Distortion Limit on Factual Hallucination Abstract Factual hallucination in closed-book question answering is often treated as a coverage problem: a model fails because the relevant fact is absent from its internal memory. This view misses a second source of error. Even when a fact has been observed, finite memory may force it to be stored only approximately. We study this effect through a simple coverage–compression model of factual recall. We consider an unstruc

发布时间：2026-09-14 12:00

抓取时间：2026-09-14 12:05

来源机构：arXiv