Rollback：在世界模型中回溯经验以改进策略
原标题：Paper page - Rollback: Experience Backtracking in World Models for Policy Improvement
AI 摘要
该论文提出 Rollback，一种利用世界模型进行策略改进的新范式：通过回滚到过去的关键决策点、分支探索反事实动作并对比结果，将反事实结果蒸馏为细粒度的动作级监督，无需单独学习价值模型。作者构建了统一分布式世界模型搜索与异步真实机器人执行的训练系统，形成真实经验持续修正世界模型、世界模型再驱动策略改进的循环。在三个真实机器人任务上，Rollback 相比最强基线成功率分别绝对提升 10.67%、11.11% 和 11.85%，并在 8 小时内于抛球和桌面冰壶任务上超过人类玩家平均成功率。
正文节选
Rollback: Experience Backtracking in World Models for Policy Improvement Abstract Rollback improves robot policies by using world models to rewind to past decisions, explore counterfactual actions, and generate fine-grained supervision, while continuously refining dynamics via real-world feedback. Action-conditioned world models offer a scalable alternative to costly real-world interaction for improving robot policies. However, existing world-model-based methods largely treat world models as for