自动化可解释性评估数据存在结构性捷径
原标题：Scoring Explanations Without Reading Them: Structural Shortcuts in Automated-Interpretability Evaluation Data
AI 摘要
该论文发现，自动化可解释性评估数据中存在结构性捷径：在Neuronpedia存储的500个GPT-2 small SAE特征模糊测试运行中，91.3%的激活样本被裁剪为峰值位于中心token，一个从不阅读解释、仅凭中心token是否被标记作答的规则即可达到0.979的平衡准确率，高于存储LLM评分器的0.838。类似线索还出现在delphi的固定标记位置、全上下文运行的标记数量以及OpenAI模拟基准的序列排序中。作者提出应把评分与同一批样本上最佳的无解释规则并列报告，并构建了去除位置线索的窗口，在自然文本窗口上Jev评分器达到0.331的合并相关性，而打乱解释仅0.013。
正文节选
Scoring Explanations Without Reading Them: Structural Shortcuts in Automated-Interpretability Evaluation Data Abstract Automated interpretability scores an explanation of a sparse-autoencoder (SAE) feature by asking a model to predict, token by token, where the feature fires on evaluation examples or to judge whether marked tokens fit it. We find that the evaluation data standard pipelines build can be scored without reading the explanation. On Neuronpedia’s stored fuzzing runs for 500 random GP