GAVAGAI：基于平衡最优传输的婴儿规模视觉语言指称对齐
原标题：GAVAGAI: Referential Alignment for Infant-Scale Vision-Language Models via Balanced Optimal Transport
AI 摘要
该论文提出GAVAGAI方法，将婴儿规模视觉-语言模型中的词-区域指称对齐建模为带容量约束的熵最优传输问题，引入“Nothing”列处理非指称词，并用容量约束编码互斥性。在受控模拟中，朴素对齐在真实非指称率下准确率崩溃至0.000，而双约束版本达到0.075并优于纯描述目标；在SAYCam婴儿视频语料上，加入对齐损失使词-图准确率从32.4%提升至48.7%，定位IoU从0.18提升至0.38。作者强调该损失改善的是高非指称率下的描述目标，而非刷新最佳基准。
正文节选
GAVAGAI: Referential Alignment for Infant-Scale Vision-Language Models via Balanced Optimal Transport Abstract A child learns which words name which things from roughly 100 million words of input. Vision-language models trained at a comparable scale bind words to referents far less reliably, and we argue that part of the gap is structural: the usual next-token captioning objective has no variable standing for “this word refers to that region,” so training applies no direct pressure to ground wor