MedProb:探测医学视觉问答中视觉语言模型的内部表示
原标题:MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering
AI 摘要
MedProb 是一个轻量级线性探测框架,通过冻结视觉语言模型(VLM)的内部表示来预测医学视觉问答(Med-VQA)的多选题答案,无需自由文本生成。在 PATH-VQA、SLAKE 和 VQA-RAD 基准上,MedProb 比提示方法恢复了更多与答案相关的信号,并优于医学微调 VLM 和智能体系统。研究发现,生成式评估可能低估小型模型中的可恢复信号,且医学适应并不一致地提高线性可解码性。此外,MedProb 对选项顺序的偏差比自由文本生成更稳定。
正文节选
Abstract Medical visual question answering (Med-VQA) is often assumed to require medical fine-tuning, large models, or complex multi-agent pipelines. We revisit this assumption with MedProb, a lightweight probing framework that predicts multiple-choice Med-VQA answers from frozen VLM representations without free-text generation. Across PATH-VQA, SLAKE, and VQA-RAD, MedProb recovers substantially more answer-relevant signal than prompting and performs stronger than medical VLMs and agentic system