返回全部动态
量化grokking中记忆到泛化的转变：缩放律与相结构
原标题：Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking
AI 摘要
该研究通过在模算术任务上对两层MLP进行384组超参数扫描，量化了grokking中从记忆到泛化的转变。研究发现泛化起始时间遵循幂律缩放关系，其中数据复杂度的影响比模型容量高一个数量级，并在权重衰减维度上存在清晰的相边界。权重范数在转变过程中单调压缩，表明隐式正则化选择了低复杂度解。
以上摘要由 AI 生成，可能存在误差。事实请以原文为准。
正文节选
Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking Abstract Neural networks trained past memorization frequently undergo a delayed transition to generalization, a phenomenon known as grokking. Despite theoretical progress on why this transition occurs, the quantitative structure of when it occurs in hyperparameter space remains uncharacterized. We map the memorization-to-generalization boundary across 384 configurations of two-hidden-layer MLP
发布时间：2026-09-12 12:00
抓取时间：2026-09-12 12:12
来源机构：arXiv