重新思考数据选择：语义流形上的分布匹配
原标题：Rethinking Data Selection as Distribution Matching on Semantic Manifolds
AI 摘要
论文提出 QMAC，一种质量感知的多频带特征函数匹配数据选择方法，将数据选择建模为分布匹配与质量过滤的联合优化问题。该方法在多个频率尺度上匹配所选子集与全量数据的经验特征函数，以同时捕捉全局语义覆盖与局部扰动，并用任务相关质量分数过滤噪声和低质量样本。作者在 4 种数据类型、8 个目标模型和 44 个基准上系统评估，结果显示 QMAC 在标准图文指令数据上优于现有数据选择基线，在文本指令、图文推理和文本推理数据上显著超过随机选择，并在多种模型与预算组合下超过全量数据训练。
正文节选
Rethinking Data Selection as Distribution Matching on Semantic Manifolds Abstract Instruction tuning of large language models and large vision language models relies on massive training data, which incurs substantial computational and storage costs. Meanwhile, training data often contains redundancy, noise, and low quality samples. Existing data selection methods mostly rely on diversity, importance, or a combination of both, without explicitly constraining the distributional consistency between