LLM 记忆缺失的一环：长上下文与检索不足以修正信念
原标题：The Missing Piece in LLM Memory: Why Long Context and Retrieval Aren't Enough for Belief Revision
AI 摘要
Hugging Face 博客发布两篇系列文章，探讨大语言模型记忆机制中「访问」与「修正」的区别。文章指出，长上下文和检索只能让模型保留并访问过去信息，但当新证据与旧信息冲突时，系统未必能正确更新记忆，这一现象被称为 context rot。作者以线性注意力为例，说明其固定大小的循环状态使记忆更新过程可被直接观察，并指出简单累加式记忆在信息被替换时会把新旧关联混合而非覆盖，进而引出 delta rule 等修正机制。
正文节选
A two-part series on how language models store, update, and revise information. TL;DR: Why LLMs Struggle When Information Changes A language model can retain a million tokens of context and still act on information that is no longer true. The problem isn't always forgetting. Sometimes, the information is still there, but the model fails to account for what has changed. This article explores the difference between keeping information accessible and revising what memory represents. - Access is not