ReRound：利用重建舍入解决免校准 LLM 量化中的中点歧义
原标题：ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
AI 摘要
ReRound 是一种无需校准的后训练量化方法，通过条件扩散模型生成低比特权重的连续重建，以解决标准舍入到最近方案在量化区间中点附近的歧义问题。该方法利用容差度量选择候选量化矩阵，并通过匹配前导奇异值确定最优容差，在 3-bit 和 4-bit 权重量化中优于标准 RTN，且不增加推理开销。ReRound 对小规模 LLM 特别有效，其性能优于多种免校准方法，并与依赖校准的方法相当。
正文节选
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization Abstract ReRound uses a conditional diffusion model to guide rounding of near-midpoint weights during low-bit post-training quantization, selecting candidates by matching leading singular values to improve small LLM accuracy without inference overhead. ReRound (Reconstructive Rounding) is a post-training quantization method that addresses the midpoint ambiguity inherent in standard round-to-neares