重新思考多模态LLM遗忘的监督目标：更少隐私信息与更小分布偏移
原标题：Rethinking Supervision Targets for Multimodal LLM Unlearning: Less Private Information, Reduced Distribution Shift
AI 摘要
该论文研究多模态大语言模型（MLLM）的隐私信息遗忘问题，指出监督目标中残留隐私信息会导致遗忘不充分，而与原始输出分布偏移过大会损害模型效用。作者提出 FFN-Perturbed Self-Distillation（FPD）方法，从教师模型原始输出分布出发，选择性扰动与隐私信息相关的 FFN 激活，从而构建隐私信息更少、分布偏移更小的监督目标。在 S-MLLMUn Bench 上的实验表明，FPD 在遗忘效果与模型效用之间取得了更好的平衡。
正文节选
Rethinking Supervision Targets for Multimodal LLM Unlearning: Less Private Information, Reduced Distribution Shift Abstract Multimodal Large Language Models (MLLMs) can memorize and reproduce sensitive information from their training data, posing significant privacy risks. Their ability to encode knowledge across both textual and visual modalities further complicates the selective removal of such private information. Many existing methods guide model unlearning by training the model toward const