ResCom：面向大语言模型量化的激活残差补偿
原标题：ResCom: Activation Residual Compensation for Large Language Model Quantization
AI 摘要
论文提出 ResCom，一种用于大语言模型低比特量化（W4A4）的激活残差补偿方法：在模型完成校准后，利用激活舍入残差，通过低秩校正来补偿层输出误差，无需梯度训练，也不改动已校准的权重。在 Llama-3-8B 上，rank-32 的注意力校正将 WikiText-2 困惑度从 7.708 降至 7.147，MMLU 提升 2.89 个百分点，GSM8K 五样本准确率从 23.96% 升至 36.16%。该方法在 1B 至 70B 参数模型及 C4 留出集上做了验证，FP32 实现使 4K prefill 时间增加 2.25%，紧凑存储实现相对 FP16 仍保留 27.7% 的峰值内存降低。
正文节选
ResCom: Activation Residual Compensation for Large Language Model Quantization Abstract Low-bit weight–activation quantization reduces the memory and computation needed to deploy large language models, but activation rounding can still degrade accuracy after calibration. Reconstruction from rounded inputs cannot distinguish activations within the same quantization bin. We propose ResCom, a low-rank correction that uses the activation rounding residual to compensate for the resulting layer-output