DLR-Lock：通过深度低秩残差蒸馏锁定预训练权重
原标题：Locking Pretrained Weights via Deep Low-Rank Residual Distillation
AI 摘要
Apple 与东京大学的研究者提出 DLR-Lock 方法，通过将预训练模型中的 MLP 替换为深度低秩残差网络（DLR-Net），利用自动微分的推理-训练不对称性，增加微调时的激活内存和计算开销，从而阻止未经授权的权重修改。该方法在保持模型能力的同时，能抵御完全了解防御策略的自适应攻击者。该论文已被 ICML 2024 的 Efficient Systems for Foundation Models 研讨会接收。
正文节选
The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling their use across diverse hardware and software platforms. They also allow for more open research and testing, to the extent that users can use them as checkpoints, fine-tune them according to their needs, and potentially redistribute them. In some cases, however, concerns on modifying these weights towards unauthorized uses may outweigh the pros of