VLM为何在远程生理测量上失败?VITAL框架提出生理接地与数值对齐
原标题:Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment
AI 摘要
该论文研究视觉语言模型(VLM)在远程生理测量(RPM)任务上的失败原因,指出两大瓶颈:通用视觉语义表示无法保留细粒度生理线索,以及自回归离散 token 预测与连续数值测量之间的错位。作者提出 VITAL 框架,保留原有语义视觉通路并新增独立生理通路生成 PhysTokens,同时用结构化数值对齐改进心率估计。实验表明 VITAL 在多个 rPPG 数据集上提升跨域估计性能,并保留 VLM 的通用视觉理解能力。
正文节选
Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment Abstract Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems. However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the multimodal understanding and interac