PRQuant：面向低开销推理的置换残差量化
原标题：PRQuant: Permutation Residual Quantization for Low-Overhead Inference
AI 摘要
该论文提出 PRQuant，一种无需训练、低开销的 W4A4 量化框架，结合通道重排与静态权重侧残差补偿。方法先用 AWQ 风格缩放识别对权重量化误差贡献最大的输入通道，将其置换到连续尾部块并离线构建残差子张量，使推理时残差补偿转化为规则的尾部增强 GEMM，避免在线收集开销。实验显示 PRQuant 在五个下游基准上平均准确率超过默认 MXFP4 及所评估的 PTQ 基线，在 Qwen3-4B-Instruct-2507 和 Qwen3-30B-A3B-Instruct-2507 上分别比 MXFP4 提升 1.24 和 0.55。
正文节选
PRQuant: Permutation Residual Quantization for Low-Overhead Inference Abstract Accuracy of Low-bit quantization of linear layers is often dominated by a small number of outliers. Although existing methods, such as smoothing, rotation, or residual-based approaches, may mitigate this problem, they often introduce new accuracy bottlenecks to weights. Besides, most of these techniques are implemented as online approaches, which can result in heavy execution overheads. To address the afore-mentioned