多模态预训练的物理机制：知识流、模态协同与早期统一

AI 摘要

该研究系统探索了多模态预训练的机制，通过合成和真实数据集实验，揭示了知识流、模态协同与竞争、早期统一等关键现象，并提出了高效预训练方案，仅用5%计算预算即可实现强生成性能。研究在13.5B MoE模型和2T tokens上验证了结论，为多模态预训练提供了原则性基础。

正文节选

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes Abstract Vision offers a critical axis for advancing foundation models, driving a shift towards natively unified multimodal pretraining. Despite this momentum, the design space and the fundamental mechanisms of how modalities interact during unified training remain underexplored. We provide empirical clarity through a systematic exploration of multimodal pretraining. Our controlled experim

发布时间：—

抓取时间：2026-08-06 20:57

来源机构：Hugging Face