前向传播域适应：无需跨层反向传播的LLM微调方法
原标题：Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation)
AI 摘要
arXiv 论文提出前向传播域适应方法 FPO，无需跨层反向传播即可微调大语言模型，吞吐量提升 2.7-3.2 倍，峰值内存降低 40%，且不损害域外基准。该方法基于输出层预测误差与真实梯度在后期层的余弦相似度 0.47-0.59 的观察，并引入两分钟诊断工具确定适用层。实验在 OLMo-2-7B、Qwen3-8B、Falcon3-7B 上验证，域内困惑度改善，MMLU 等基准保持基线水平。
正文节选
Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation) Abstract Forward-Pass-Only MLP training (FPO) adapts large language models without a backward pass through the model body, achieving 2.7–3.2 the throughput of standard fine-tuning at 40% less peak training memory, while leaving off-domain benchmarks within seed-noise of baseline, a property that full-network fine-tuning does not reliably reproduce. FPO rests on a single empirical observation: at late layers of a transformer, th