链式思考何时有效：LLM推理中序列深度瓶颈的实证研究
原标题：When Chain-of-Thought Helps and When It Hurts: An Empirical Investigation of the Serial-Depth Bottleneck in LLM Reasoning
AI 摘要
该研究通过实证调查挑战了链式思考（CoT）提示普遍提升大语言模型推理能力的假设。基于带宽约束框架，研究在三个指令微调模型和五个基准测试中发现，CoT的效果取决于任务的序列深度：在高深度任务（如GSM8K、MATH）上CoT带来显著提升，而在低深度任务（如MMLU、ARC-Challenge）上CoT几乎无益。研究结论表明，CoT并非通用的推理增强器，而是作为带宽旁路，仅对超出单次前向传播能力的序列计算有帮助。
正文节选
When Chain-of-Thought Helps and When It Hurts: An Empirical Investigation of the Serial-Depth Bottleneck in LLM Reasoning Abstract It is widely assumed that chain-of-thought (CoT) prompting universally improves LLM reasoning. We investigate this assumption through the conceptual framework of the bandwidth bound (Chen et al., 2024). While the formal bound applies only asymptotically – at astronomically large prompt lengths – it identifies a fundamental architectural bottleneck: serial computation