组合持续学习机制实现长时程记忆
原标题：Paper page - Continual Learning Mechanisms Compose for Long-Horizon Memorization
AI 摘要
该论文提出「长时程记忆」设定：模型通过持续监督微调依次学习100个问答任务，且不保留旧训练样本、推理时也不提供任务标识。作者将数据锚、函数锚、权重锚与合并式低秩更新（merged LoRA）组合，并构建三个100任务数据集，用任务级逐次减半搜索与因子实验评估。最佳方法在三个数据集上均进入前三，将平均最终保留率从朴素顺序微调的1.2%提升至34.9%，约28倍，其中数据锚与merged LoRA增益最大且呈超加性交互。
正文节选
Continual Learning Mechanisms Compose for Long-Horizon Memorization Abstract Composing complementary continual learning mechanisms substantially improves long-horizon memorization in language models by combining data, function, and weight anchors with merged low-rank updates. Language models may need to internalize information that arrives over time and retain it through many subsequent updates. To study this challenge, we introduce long-horizon memorization, a setting in which a model learns 10