平滑Transformer前馈网络的曲率密码分析
原标题：Curvature Cryptanalysis of Smooth Transformer Feed-Forward Networks
AI 摘要
该研究提出了一种针对Transformer前馈网络（FFN）的曲率密码分析攻击方法，在仅能进行黑盒查询（选择输入、获取原始输出）的条件下，通过分析投影输入Hessian矩阵，恢复隐藏的FFN第一层权重方向。实验表明，在CIFAR-10视觉Transformer上，该方法能以高对齐度恢复方向，并支持功能提取，构建高保真替代模型。输出舍入和高斯噪声会降低恢复效果，但调整有限差分步长可恢复性能。
正文节选
Curvature Cryptanalysis of Smooth Transformer Feed-Forward Networks Abstract We show that smooth two-layer feed-forward networks (FFNs) expose an additional structural model-extraction channel under a chosen-input raw-output oracle at the FFN branch. We consider transformer FFN branches with GELU or SiLU activations under chosen-input raw-output access, without access to parameters, gradients, or internal activations. We characterize and exploit a second-order leakage channel in which projected