KVBoost：块级 KV 缓存复用与偏差引导重计算加速 LLM 推理
原标题：KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference
AI 摘要
KVBoost 是一个面向 HuggingFace 兼容解码器模型的块级 KV 缓存复用系统，通过双哈希键控和偏差引导的重计算，解决了共享内容出现在提示任意位置时的缓存命中问题。在 Qwen2.5-3B 模型上，KVBoost 相比全量重计算实现了 4.49 倍的 TTFT 加速，比 vLLM 前缀缓存快 16%，且无质量损失。该系统支持 KV 量化、磁盘溢出缓存和自适应分块，可作为生产就绪的推理加速层。
正文节选
KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference Srihari Unnikrishnan Independent Research srihari.unnikrishnan@gmail.com Abstract Transformer-based large language models (LLMs) incur significant prefill latency when processing long or repeatedly-shared prompt prefixes, because key-value (KV) tensors must be recomputed in full for each request. Existing prefix-caching systems mitigate this cost but require prompts to shar