LLM 偏见评估中人类衍生构念的批判
原标题：Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluation
AI 摘要
这篇 arXiv 论文对当前 LLM 偏见评估中直接套用人类偏见构念（如内隐联想测验、锚定效应、框架效应等）的做法提出批评。作者指出，心理测量工具原本用于研究人类认知与社会行为，而 LLM 评估依赖 token 概率、文本补全、排序或模拟决策，二者之间存在推断鸿沟。论文提出一个分析框架，用于厘清不同评估设计能支持何种结论，并强调需要更强的理论依据。
正文节选
Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluation Abstract Researchers increasingly use human-derived bias constructs to study Large Language Models (LLMs), including social-cognitive constructs such as implicit bias and stereotype activation, and cognitive biases such as anchoring, framing effects, and confirmation bias. Such approaches offer alternatives to overt bias probes, particularly when direct questioning may obscure bias or when model beh