大型语言模型思维链推理的平均场动力学
原标题：Mean-Field Dynamics of Chain-of-Thought Reasoning in Large Language Models
AI 摘要
该研究提出一个框架，在不简化模型架构或类比物理系统的情况下，为大型语言模型的思维链推理寻找统计规律和理论解释。他们将推理建模为线索图上的引导发现过程，利用平均场近似推导出已发现线索比例的常微分方程。实验通过学生模型对教师模型输出的归一化惊讶值识别线索标记，并跨多条推理链平均获得统计规律，结果显示这些规律在同一数据集内可复现，并能用所提出的理论方程拟合。
正文节选
Computer Science > Computation and Language Title:Mean-Field Dynamics of Chain-of-Thought Reasoning in Large Language Models View PDF HTML (experimental) Abstract:Large language models (LLMs) with chain-of-thought reasoning have been widely applied in recent years, and theoretical explanations of their behavior may help deepen our understanding and guide model optimization. In this study, we introduce a framework that seeks statistical regularities and theoretical interpretations in