TopKSigLIP：可微子采样解决乳腺X线摄影VLM的“大海捞针”问题
原标题：Solving the Needle-in-a-Haystack Problem in Mammography Vision-Language Model with Differentiable Subset Sampling
AI 摘要
本文提出TopKSigLIP，一种用于乳腺X线摄影的视觉-语言模型，通过TopK-Patch模块学习采样高分辨率补丁以解决高分辨率与稀疏病灶问题，并采用Sup-sigmoid损失替代对比损失以应对报告同质性。该模型在零样本评估中优于现有开源乳腺和通用医学VLM，在密度评估、BI-RADS分类、发现亚型和癌症预测等任务上表现更佳，且TopK-Patch模块的病灶定位优于Grad-CAM。代码和权重已公开。
正文节选
Solving the Needle-in-a-Haystack Problem in Mammography Vision–Language Model with Differentiable Subset Sampling Abstract There is growing interest in adopting CLIP-style vision–language model (VLM) pretraining for mammography. However, models that directly employ the standard CLIP architecture and training objective exhibit limited zero-shot performance in clinically important tasks such as cancer, finding-type, and BI-RADS predictions. We argue that this underwhelming performance is due to ne