DreamTraj：通过读取未渲染的视频扩散潜变量生成 6-DoF 物体轨迹
原标题：DreamTraj: Generating 6-DoF Object Trajectories by Reading Unrendered Video Diffusion Latents
AI 摘要
DreamTraj 提出一种新方法，从单张 RGB 图像和语言指令直接预测物体的 6-DoF 轨迹，通过读取冻结的图像到视频扩散模型在早期去噪步骤中的内部表示，而非生成视频后再提取运动。该方法还引入了 MOVE 数据集，包含 5,038 个带细粒度语言注释的以物体为中心的自我中心轨迹。DreamTraj 在平移和旋转预测上均达到最先进水平，且比生成后提取的流程快 4.6 倍。
正文节选
DreamTraj: Generating 6-DoF Object Trajectories by Reading Unrendered Video Diffusion Latents Abstract Accurate prediction of object trajectories during manipulation is essential for closing the perception-action loop. Progress is limited on two fronts: available datasets lack fine-grained language-to-motion annotations, and existing predictors either rely on privileged inputs such as video, depth, or CAD models, or recover motion from fully generated videos through costly, error-prone perceptio