逆推理模型：仅用答案数据的高效后训练与验证
原标题：Inverse Reasoning Models for Efficient Post-Training and Verification from Answer-Only Data
AI 摘要
该论文提出一种逆推理模型（inverse reasoning LLM），将思维链（CoT）视为隐变量，在仅有答案的问答数据上训练一个以答案和问题为条件生成推理轨迹的辅助模型，从而实现高效的离线自蒸馏后训练。在 Qwen3-0.6B 上针对 Countdown 和 GSM8K 任务，GRPO 训练饱和后经一轮离线蒸馏，困难提示的 pass@1 约翻倍，并在 Countdown 上全面优于拒绝采样自蒸馏。该逆模型还可作为正确性验证器，在测试时 best-of-n 选择中将 Countdown 的选择准确率从 0.753（多数投票）提升至 0.969。
正文节选
Inverse Reasoning Models for Efficient Post-Training and Verification from Answer-Only Data Abstract Post-training reasoning large language models (LLMs) often requires making a tradeoff between costly supervision and compute inefficiency: supervised finetuning typically requires reasoning traces, which is data inefficient since in many applications one only has prompt-answer tuples and lacks the groundtruth chain-of-thought (CoT) sequence. On the other hand, reinforcement learning (with verifia