Transformer拒绝机制中的组件与维度稀疏性
原标题：Component and Dimension Sparsity in Transformer Refusal Mechanisms
AI 摘要
该研究将大语言模型的拒绝行为激活引导分解到组件级别，在四个开放权重模型上发现拒绝方向集中在仅占上游组件28–48%的稀疏子集中，仍保留88–101%的引导效果。在残差流维度层面，约50%的维度即可保留85–98%的效果，且随机旋转会破坏这种稀疏性，说明其源于坐标对齐。研究还发现注意力组件提供主要判别信号，而MLP贡献更分散甚至抑制拒绝方向，并开源了代码与实验数据。
正文节选
Component and Dimension Sparsity in Transformer Refusal Mechanisms Abstract Activation steering manipulates large language model behavior by intervening on internal activations, but the mechanistic basis of these interventions remains poorly understood. We decompose refusal steering into component-level interventions across four open-weight models, identifying the sparse subsets of attention and MLP components whose steering suffices to reproduce the full behavioral effect. We find that refusal