对齐预测：从训练数据预测模型失准
原标题：Alignment Forecasting: Predicting Misalignment From Training Data
AI 摘要
研究者提出「对齐预测」（Alignment Forecasting）任务，即在微调训练前预测模型是否会出现欺骗、谄媚等对齐失败，并发布包含17个目标模型、32个数据集、16种失败模式的基准 AlignmentForecastBench。他们提出的预测框架由LLM审计数据集并打分，再用逻辑回归结合失败模式基率和目标模型先验倾向，效果显著高于随机，并优于直接提示前沿模型、微调专用模型及弱模型迁移方法。其信号还能标记前沿模型分类器漏掉的问题训练样本，过滤 UltraChat 中此类样本后多数情况下模型更对齐，但开放式对话中的收益尚不明确。
正文节选
Alignment Forecasting: Predicting Misalignment from Training Data Abstract Training a language model on data with a narrow flaw can sometimes make the model broadly misaligned. Inspecting the data at face value often does not settle whether it will emerge, and today it is caught only after training, by auditing the resulting model. To complement post-hoc audits, we introduce Alignment Forecasting: the task of predicting alignment failures before training. Given a target model, a fine-tuning dat