基于状态空间编码的线性时间上下文压缩
原标题：Linear-Time Context Compression with State-Space Encoding
AI 摘要
研究提出 Boa Compressor，一种基于 Mamba 状态空间模型的即插即用在线上下文压缩器，可在单次线性时间遍历中编码任意长度上下文，并以此条件化冻结的目标 LLM。该压缩器通过蒸馏训练，使压缩表示下的 LLM 输出分布匹配完整上下文下的教师分布。在 MRQA 基准上，同等压缩比下 Boa 匹配或超越现有最优离线压缩基线，速度提升一个数量级，并支持高达 16 倍的激进压缩比，使长上下文推理的在线压缩变得实用。
正文节选
Linear-Time Context Compression with State-Space Encoding Abstract Transformer-based large language models (LLMs) impose substantial computation and memory costs as context sizes increase, motivating methods that compress a context into a shorter representation before inference. Existing soft-prompt compression methods typically use the same Transformer backbone as used in the target model, where producing the compressed representation leads to significant latency overheads. This restricts them