DoSA：点稀疏注意力，无损加速稀疏注意力索引器
原标题：DoSA: Dots Sparse Attention
AI 摘要
论文提出 DoSA（Dots Sparse Attention），一种无损稀疏注意力方法，通过为每个 token 构造分数上界与 Top-k 阈值下界，在保留精确 token 选择的前提下跳过大量全头打分。该方法针对 DeepSeek 稀疏注意力（DSA）中带符号 ReLU 索引器的稠密打分瓶颈，利用非负头投影、联合残差界和激活感知聚合来收紧候选排除条件。在 GLM-5.2 与 DeepSeek-V3.2 上，约 100 万 token 上下文时端到端与首 token 延迟相比 SGLang 等基线最高提升近一倍，索引器耗时占比最高达 90.8%。
正文节选
DoSA: Dots Sparse Attention Abstract Sparse attention visits few keys, yet finding them can still require scoring every key with every indexer head. We introduce DoSA (Dots Sparse Attention), a lossless method for making this computation conditional while preserving exact token selection. Its central observation is that Top- selection requires sufficient evidence to reject a candidate, not necessarily its full score. A seed set supplies a lower bound on the winning threshold. Nonnegative head pr