修辞如何影响AI评审?研究揭示结构化偏差
原标题:How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review
AI 摘要
一项研究通过构建4200篇改写论文和超过42000次AI评审,系统分析了修辞框架对AI科学评审分数的影响。研究发现,证据框架和新颖性立场对评分影响最大,且效果受评审者身份、原始分数和评审严格程度影响,而非改写复杂度。研究还发现,更复杂的改写流程并不总能带来更大收益,并建议构建对内容保持性写作变化鲁棒的评估系统。
正文节选
How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review Abstract Rhetorical framing significantly biases AI scientific review scores in structured ways, with effects shaped by reviewer identity, score range, and evaluation strictness rather than rewriting complexity. As large language models increasingly participate in scientific evaluation, we investigate a potential form of reward hacking: how rhetorical choices shape AI-review judgments when report