覆盖度而非难度决定激活探针所需合成数据量
原标题：Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs
AI 摘要
该研究系统测量了激活探针（activation probe）在监控大语言模型时所需的合成数据量，覆盖高风险情境、有害回复和指令遵循三类监控概念。研究发现，探针所需样本量主要由监控概念本身决定，而非单类样本的难度；高风险和有害探针在80个样本后即接近平台期，而指令探针需要数倍样本。核心机制是“覆盖度”：不同类别样本之间的迁移程度决定了概念间的数据需求差异，高风险概念下各类别几乎完全互相覆盖，指令概念下则几乎不覆盖。研究发布了评估套件、开发集和生成集。
正文节选
Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs Abstract Activation probes that monitor deployed language models are trained on synthetic conversations, and how many a probe needs is open. We trace learning curves over 10–590 synthetic samples for three monitoring concepts—high-stakes situations, replies harmful to a person, and replies that do not follow the user’s instruction—on fourteen held-out evaluation distributions and four probe models, varying the gener