冻结解码器，修复编码器：基于SVD的KV缓存压缩参数高效适配
原标题：Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compression
AI 摘要
该论文研究基于SVD的KV缓存压缩中的参数高效微调方法，发现共享学习率会导致不公平比较，使最小可训练参数臂看似占优。在Qwen2.5-VL-3B-Instruct上，当每个实验臂使用各自调优的学习率后，仅训练编码器（冻结解码器）与仅训练解码器或全量微调达到相同精度，同时减少3个可训练参数和3倍优化器状态内存（0.57 vs 1.72 GiB）。该结果在纯文本测试平台的两个骨干网络上得到复现，表明仅编码器修复是一种低内存的KV缓存压缩适配方案。
正文节选
Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compression Abstract Comparing parameter-efficient fine-tuning recipes under a single, shared learning rate is a common but flawed practice: when the arms being compared have very different trainable-parameter counts, a shared rate can simultaneously depress the larger arms’ means and inflate their variance, manufacturing a large, seemingly multi-seed-significant advantage for the smallest arm that is no