CoDR:面向扩散语言模型的无训练置信漂移重掩码
原标题:CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
AI 摘要
论文提出 CoDR(Confidence-Drift Remasking),一种无需训练、与采样器无关的扩散语言模型精修方法。该方法通过 k-partition probing 在少量前向传播中估计已提交 token 的置信度漂移,将模型不再支持的 token 重新掩码并重新生成。在 LLaDA-8B 和 Dream-7B 两个骨干模型、四个推理与编码任务、三种基础采样器上,CoDR 提升了平均准确率并改善了准确率-计算权衡。
正文节选
CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models Abstract Masked diffusion language models (MDLMs) decode by repeatedly committing tokens to masked positions, but these commitments are usually irreversible. A token chosen under sparse, partial context is kept fixed, even when later context no longer supports it. Existing samplers mainly decide when to commit a token, but rarely check whether an already committed token should still be kept, allowing early mistakes to