临界深度平衡模型的响应重整化
原标题：Response Renormalization for Critical Deep Equilibrium Models
AI 摘要
本文提出了一种名为响应重整化（Response Renormalization）的框架，用于改善深度平衡模型（DEQ）的训练稳定性。该方法通过在后向传播中提升选定的近极点分母，同时保留未提升的响应通道，从而控制临界邻域内的伴随放大。实验表明，在23个多物理场系列中，使用该方法训练的模型在超过98%的静态和95%的瞬态比较中，测试误差与精确隐式微分训练的模型相差不超过5%。
正文节选
Response Renormalization for Critical Deep Equilibrium Models Abstract Deep Equilibrium Models (DEQs) compute predictions by finding a hidden representation that remains unchanged under the model’s update. Training through this equilibrium uses implicit differentiation, which requires solving an adjoint system built from the residual Jacobian. If this Jacobian is nearly singular along directions to which the loss is sensitive, small perturbations can be strongly amplified in the adjoint response