自监督预训练提升多模态卒中复发预测的跨模态学习
原标题：Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework
AI 摘要
该研究探讨了自监督图像预训练与选择性参数冻结策略对多模态卒中复发预测的影响。研究基于491名患者的3D CTA影像与临床表格数据，预训练并微调了ResNet和ViT两种多模态网络，并与先前基线及从头训练模型对比。结果显示自监督预训练能更有效利用多模态数据，最佳ViT模型克服了单模态崩溃，且视觉与性别、CHD之间存在显著跨模态交互。代码已公开。
正文节选
Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework Abstract Multimodal stroke recurrence prediction requires effective integration of heterogeneous clinical and imaging data, yet modality imbalance often causes models to over-rely on dominant modalities and underutilize complementary information. While self-supervised pretraining and selective parameter freezing are commonly employed to improve representation learning and fin