V-CoLA：基于线性注意力的视觉Token压缩
原标题：Paper page - V-CoLA: Vision Token Compression with Linear Attention
AI 摘要
该论文提出 V-CoLA，一种面向线性注意力架构（如 Qwen3.5）的免训练视觉 token 压缩框架。现有基于 softmax 注意力的压缩方法在混合线性注意力模型上性能明显下降，V-CoLA 通过基于循环状态的唯一性感知重要性评分与自适应 token 合并来解决这一问题。实验显示，仅保留 50% 视觉 token 时可维持原性能的 99.5%，保留 12.5% 时仍超过 88%，并带来 1.86 至 6.15 倍的 prefill 加速。
正文节选
V-CoLA: Vision Token Compression with Linear Attention Abstract Vision-language models (VLMs) have demonstrated impressive capabilities but suffer from substantial computational overhead, as vision tokens dominate the input sequence. This motivates vision token compression as a key direction to alleviate the burden. However, with the emergence of hybrid architectures incorporating linear attention (\eg, Qwen3.5), prior methods designed for softmax attention struggle to generalize. Our analysis r