扩散模型微调中LoRA秩的权衡研究
原标题：Understanding LoRA Rank Trade-offs in Diffusion Model Fine-Tuning
AI 摘要
该研究对扩散模型微调中的LoRA秩选择进行了受控实验，在CIFAR-10上使用DDPM U-Net，固定优化设置，比较不同秩下的FID、可训练参数量、运行时间和GPU内存。结果显示中等秩（rank 4和8）在固定训练预算下效率最高，更高秩带来的质量提升有限。研究还用20轮DDPM和Tiny DiT骨干验证了趋势的稳健性，为实践者提供了小到中等秩作为默认选择的经验依据。
正文节选
Understanding LoRA Rank Trade-offs in Diffusion Model Fine-Tuning Abstract Selecting LoRA rank for diffusion fine-tuning requires balancing quality and compute cost. We present a controlled study on CIFAR-10 using a DDPM U-Net with ranks , fixed optimization settings, and a reproducible local-folder pytorch-fid protocol. We report FID, trainable parameters, runtime, and GPU memory, then validate trends with extended-budget DDPM runs (20 epochs; ranks 4/8/16) and a Tiny DiT backbone (10 epochs; r