Modular TTT：将测试时训练重构为可组合模块
原标题：Modular TTT: Rethinking Test-Time Training as Composable Modules
AI 摘要
Modular TTT 提出了一种将测试时训练（TTT）表示为可组合模块的框架，通过有向无环图表示内部学习器，并将快速权重网络、损失函数、学习率、权重衰减和归一化作为显式设计维度。系统消融发现，小的学习率初始化、权重衰减和单层非线性可提升性能，而更深的快速权重网络和归一化会因激活值过大而损害性能。基于这些发现训练的 410M 和 1.45B 参数模型在 100B token 上表现与 Gated DeltaNet 相当。
正文节选
Modular TTT: Rethinking Test-Time Training as Composable Modules Abstract Test-time training (TTT) views sequence modeling as an online learning problem in which fast weights are updated by an internal learning rule. Despite the growing number of TTT variants, existing approaches typically hard-code each variant separately, which makes it difficult to design new TTT methods and to isolate the role of each component. To address this, we propose Modular TTT, a framework that represents the inner l