子空间节点剪枝：基于正交子空间投影的免重训练网络压缩
原标题：Subspace Node Pruning
AI 摘要
该论文提出 Subspace Node Pruning（SNP），一种无需重训练的神经网络结构化剪枝方法。作者证明激活类、Fisher 类等多种重要性度量共享同一二次型，通过下三角基变换构造剪枝子空间，并用线性最小二乘同时补偿被剪节点的影响；对层激活做正交化后，各节点重要性分数解耦可加，从而支持层内多节点同时剪枝而不互相干扰。在 ImageNet 上 VGG-16 与 ResNet-50 优于对比基线，在 OPT-125M 和 LLaMA-2 7B 上 40-shot 设置下与 LLM Surgeon 持平，更少 shot 时更优，且剪枝后无需任何重训练。
正文节选
Subspace Node Pruning: Retraining-Free Neural Network Compression via Orthogonal Subspace Projections Abstract Node pruning is the art of removing computational units such as neurons, filters, attention heads, or even entire layers to significantly reduce inference time while retaining network performance. In this work, we show that a broad family of importance measures, from activation-based to Fisher-based, share a common quadratic form. Through a lower-triangular change of basis, this form yi