DramaGrow：审计多集剧 LLM 评审的 regime-aware 基准
原标题：DramaGrow: A Regime-Aware Benchmark for Auditing LLM Judges of Multi-Episode Drama
AI 摘要
北京理工大学与黑龙江大学的研究者提出 DramaGrow，一个面向 AI 生成中文五集剧的 regime-aware 基准与审计协议，将可验证约束违规、五分之四偏好和评审组内分歧分入不同分母与证据规则。基准包含 800 集、120 个字节保持干预和 1,155 条盲评五人记录。在冻结集上，重复绝对评分 rubric 的偏好准确率仅 53.8–55.8%，而 DirectPair–GLM 达到 85.1% 偏好准确率和 89.5% 约束 F1，表明评估协议、平局校准与评审边际对结果的影响不亚于 rubric 设计。
正文节选
DramaGrow: A Regime-Aware Benchmark for Auditing LLM Judges of Multi-Episode Drama Abstract LLMs can now draft complete multi-episode dramas, and LLM judges are increasingly used to evaluate them. Assessing the reliability of these judges is difficult because drama quality encompasses specification-backed errors, reproducible preferences, and legitimate disagreement among human raters. Most existing evaluations reduce these cases to a single score from one judge call and leave tie calibration, c