AdaSparse：可学习软Top-K与逐词阈值的高效稀疏检索优化
原标题：Adaptive Sparsity Optimization with Learnable Soft Top-K and Per-Term Thresholding for Efficient Retrieval
AI 摘要
该论文提出 AdaSparse 自适应稀疏化方案，通过可学习的软 top-k、逐词阈值与 FLOPs 正则化协同优化，解决 Lion-SP 等 LLM 稀疏检索模型因 LLaMA-3 大词表导致向量过长的问题。在 MS MARCO 与 BEIR 上实验显示，查询和文档长度缩减 4-5 倍，检索延迟降低 3.8 倍，存储降低 4.2 倍，同时保持有竞争力的相关性。
正文节选
Adaptive Sparsity Optimization with Learnable Soft Top- and Per-Term Thresholding for Efficient Retrieval Abstract. Recent work on neural sparse retrieval has demonstrated strong relevance by leveraging Large Language Models (LLMs) for semantic term expansion. However, learned models paired with previous sparsification techniques still yield overly long document and query vectors partly due to a large LLM vocabulary, imposing a serious challenge to retrieval time and space efficiency. This paper