有限Newton-Schulz的平滑效应：Muon在非光滑非凸优化中的优势
原标题：Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization
AI 摘要
本文研究了Muon优化器中有限Newton-Schulz迭代在非光滑非凸优化中的作用。作者通过在线到非凸转换分析，证明有限Newton-Schulz迭代将不连续的极分解映射平滑为Lipschitz映射，从而使得Muon能够收敛到驻点，而精确极分解更新可能无法收敛。该结果首次表明有限Newton-Schulz迭代是有益的而非近似误差，并扩展至一般谱映射。
正文节选
Muon with Finite Newton–Schulz: The Smoothing Benefit in Nonsmooth Nonconvex OptimizationThanks: Authors are listed in alphabetical order. Abstract Muon has emerged as a strong optimizer for the matrix-valued parameters in large language model pretraining, approximately orthogonalizing its momentum with a few Newton–Schulz iterations. Existing theory either replaces this iteration with the exact polar factor it approximates, or treats its finite depth as an approximation error, and thus the iter