WorldBridge:连接视频与动作,提升世界动作模型的任务泛化
原标题:Paper page - WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
AI 摘要
该论文提出 WorldBridge,一种面向机器人学习的 World Action Model(WAM),通过独立的视频专家与动作专家、结构化注意力与联合 flow matching,将视频预测与稀疏动作监督结合,以提升任务泛化能力。在 RoboTwin 2.0 上,仅用分布内视频-动作监督时,分布外任务成功率为 25.07%;加入无动作标签的任务视频后升至 55.90%,每个任务再加一条带动作标签的示范后达到 70.70%。WorldBridge 在多个操作基准上取得对比方法中的最优性能,包括 RoboTwin 2.0 全数据设置 96.23%、RoboCasa GR1 TableTop 82.8% 和 DOMINO Level-1 57.6%。
正文节选
WorldBridge: Bridging Video and Action for Task Generalization in World Action Models Abstract WorldBridge improves robot task generalization by combining video prediction with sparse action supervision via separate video and action experts, achieving state-of-the-art results across manipulation benchmarks. Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) throug