将VLM智能迁移至机器人控制

AI 摘要

该论文提出 RoboDawn，一种通过离散平移、旋转和夹爪命令将视觉语言模型（VLM）接入机器人控制的接口，使 VLM 能以闭环方式观察、推理并执行动作。作者引入上下文学习（ICL）方案，用少量演示让 VLM 掌握接口使用与任务策略。在 RoboTwin 2.0 C2R 和 RoboDojo 基准上，零样本即超越多个专用训练策略，单样本演示后成功率分别从 53.2% 提升至 73.6%、从 35.67% 提升至 47.17%，并超过基线 π0.5（46.0%），框架还迁移到 Franka 真实机器人完成积木入篮与堆叠任务。

正文节选

Transferring the Intelligence of VLMs to Robotic Control Abstract Humans can seamlessly adapt to both physical and digital worlds, suggesting that while a digital-to-real gap exists in embodiment, environment and task, human intelligence itself may transfer across this gap. This naturally raises a fundamental question: can the intelligence of vision-language models (VLMs) similarly generalize from the digital world to the physical world for robotic control? We investigate this question through R

发布时间：—

抓取时间：2026-09-22 18:28

来源机构：Hugging Face