RLTL;DR：通过内化自生成反馈实现自我提升
原标题：RLTL;DR: Self-Improvement by Internalizing Self-Generated Feedback
AI 摘要
Apple 研究团队提出 RLTL;DR 方法，让策略在每次失败后根据验证器输出自行撰写一条 TL;DR 式反馈，并将后续 rollout 条件化于此前所有反馈，同时通过上下文内反馈的反向传播内化「任务→洞察」映射。在 Pass@128=0 的困难工具调用与编程数据集上，标准 GRPO 训练 Qwen 3.5 9B Thinking 的 Pass@1 仅 0%–1%，而 RLTL;DR 在训练时达 14–31%，评估时无洞察上下文仍达 12–13%。作者进一步提出 SFTL;DR，仅用 4k 条（任务, 洞察）元组训练即可恢复接近完整性能。
正文节选
RLTL;DR: Self-Improvement by Internalizing Self-Generated Feedback AuthorsMichael Kirchhof, Eleonora Gualdoni, Andrew Szot, Khashayar Gatmiry, Aryo Lotfi, Abbas Kazerouni, Omar Attia, Sanjoy Chowdhury, Alexander Toshev The common paradigm of reinforcement learning with verifiable rewards (RLVR) is to let agents make multiple attempts at a task, and optimize towards the successful ones. This becomes problematic in the realms of self-improvement, where tasks are so difficult that the agent has a l