RLVR域外增益有多少来自答案格式？
原标题：Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?
AI 摘要
该论文研究RLVR（可验证奖励强化学习）在域外基准上的增益有多少来自答案格式变化而非实际能力提升。作者用固定提取器重新评分四个已发布的7B SimpleRL基座-RL模型对，发现MMLU、MMLU-Pro和ARC-Challenge上82%的域外多选题增益是格式增益——基座模型本已在文本中给出答案，RL只是把答案移入\boxed{}中，而公开代码的提取器优先读取该框。独立LLM读者给出的比例为77%，说明报告增益部分取决于提取器本身。
正文节选
Same Answer, New Box: How Much of RLVR’s Out-of-Domain Gain Is Answer Format? Abstract Reinforcement learning with verifiable rewards (RLVR) is credited with gains beyond the domain its rewards check, such as higher MMLU accuracy after training on mathematics. Such accuracies are usually read out of the model’s text by an answer extractor. A model that learns to write its answer where the extractor looks can therefore score higher without answering better. We measure how much of the gain reporte