LARA：残差流中的轻量适配器实现可组合适配与对齐
原标题：LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment
AI 摘要
arXiv 论文提出 LARA（轻量级加性残差适配）方法，在冻结模型的残差流中添加低秩校正，而非修改权重。在代码微调和 DPO 偏好优化任务上，LARA 在相同参数数量下匹配 LoRA 性能，并支持推理时通过缩放因子 γ 平滑插值，以及多行为按 token 自动路由。在 1.5B 模型上同时驻留 7 种行为，仅需约 33 MB 额外开销。
正文节选
Computer Science > Machine Learning Title:LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment View PDF HTML (experimental) Abstract:We present LARA (Lightweight Additive Residual Adaptation), a method for efficient adaptation that operates in the residual stream of a frozen model rather than in its weights. Where LoRA adds an update of low rank to weight matrices, LARA reads the hidden state at a small set of layers and adds a correction of low r