输出感知块影响：用于深度剪枝的雅可比透镜加权

AI 摘要

该论文提出一种基于雅可比透镜（J-Lens）的深度剪枝评分方法 J-BI，将 ShortGPT 的 Block Influence 局部相似度通过 J-Lens 传输到最终残差流或 logits 上再计算。在 Qwen3-8B、Llama-3.1-8B-IT 和 Gemma-3-12B-IT 上，相同稀疏度下 J-BI 相比 BI 将剪枝后准确率最高提升 19 个百分点，并降低困惑度损伤。作者还通过单块消融和 J-Lens 探针解释了性能提升的原因。

正文节选

AXIOM Output-Aware Block Influence: Jacobian-Lens Weighting for Depth Pruning Abstract Depth pruning removes entire transformer blocks and therefore depends on a reliable criterion for identifying which blocks can be removed. Block Influence (BI) ranks blocks by how minimally they rotate the hidden state. However, local redundancy does not necessarily imply downstream irrelevance. We address this limitation using the Jacobian Lens (J-Lens) neural artifact, a map transporting perturbations at a l

发布时间：2026-11-03 13:00

抓取时间：2026-11-03 17:05

来源机构：arXiv