DiffusionART：高效稳定可扩展的扩散模型强化学习
原标题：Paper page - DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models
AI 摘要
该论文提出 DiffusionART，一种针对扩散模型的在线强化学习微调方法，通过将每个提示的奖励聚合为共享的奖励改进方向，从构造上消除前向-反向一致性力，并加入信任域掩码的第二次更新以实现漂移自适应恢复，无需 EMA 调参。方法无需可微奖励模型，在更少样本下仍有效，并支持成对奖励模型（DiffusionART-Pairwise）。在图像与视频生成任务上，DiffusionART 无需逐任务调参即可高效稳定训练；在 HunyuanVideo-13B 上，其达到 DiffusionNFT 和 Flow-GRPO-Fast 后期平均奖励水平所需 GPU 小时分别减少 12.02 倍和 11.72 倍，平均训练奖励分别高出 28.8% 和 53.0%。
正文节选
DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models Abstract Online reinforcement learning (RL) fine-tunes diffusion models with black-box rewards, but reverse-process methods such as Flow-GRPO are limited in efficiency by poor credit assignment, while forward-process methods such as DiffusionNFT are efficient yet unstable without carefully tuned exponential-moving-average (EMA) off-policy training. We show that NFT-style objectives contain a hidden forward--reverse co