路径条件注意力迁移：SegFormer 从深度到执行枢纽
原标题：Path-Conditioned Attention Mobility: From Depth to Execution Hubs in SegFormer
AI 摘要
该研究以 SegFormer-B5 人像分割模型为对象，探究层级视觉 Transformer 中的注意力头能否在深度方向上迁移。研究发现，部分任务相关注意力头可在相邻深度间迁移，但迁移效果高度依赖其计算路径，仅迁移注意力头参数不够，还需适配周边表示边界，并保留供体特有的归一化与空间降采样预处理。基于此，作者将 120 个 Stage-3 注意力头从 40 个执行深度重组为 20 个执行枢纽，并引入 FOLDED_VEC 重参数化，在 NVIDIA L4 上端到端延迟降低约 3.15%–5.28%，mIoU 无显著变化。
正文节选
Path-Conditioned Attention Mobility: From Depth to Execution Hubs in SegFormer Abstract Transformer compression commonly reduces attention cost by pruning heads, dropping layers, or sharing computations across depth. These approaches often couple two distinct quantities: the amount of attention capacity that must be retained and the number of sequential depths at which that capacity must be executed. We study whether useful attention functions in a hierarchical vision transformer are instead fun