同源不同判:文本生产条件干扰俄语机器生成文本检测
原标题:Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian
AI 摘要
该研究在俄语语料(1882 篇文档)上固定文本来源,仅改变任务措辞等生产条件,用四种机器文本检测方法(可解释指数、训练分类器、零样本指标、评判模型)打分。结果显示四种方法对同一条件变化的评分方向不一致,且仅用来源标识符这一单特征即可达到中位 AUROC 1.0000,说明来源类别与语料组装方式混淆。作者据此指出,机器文本检测分数反映的是特定测量系统与语料的属性,而非文本来源的直接测量。
正文节选
Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian Аннотация Automated assessment of whether a text was machine-generated is applied where the verdict carries a cost for the author, yet origin is a property of the production process, not an observable property of the text. The conditions under which a text was produced — how rigid the task was, the editorial rules, the platform template — push it in the same direction in which a gener