返回全部动态

停止向量:将因果干预内化以实现高效推理

原标题:The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning

arXiv cs.LG一手来源研究质量 87

AI 摘要

该研究在 DeepSeek-R1-Distill-Qwen-7B 模型中发现了一个“停止向量”(halt vector),位于第 18 层,通过干预该方向可控制模型思考长度。研究者将该因果干预内化到模型权重中,从 24 个问题中拟合,无需强化学习,在五个未见基准上以保持准确率的方式减少了约 24% 的思考,并解决了非终止病理问题。

以上摘要由 AI 生成,可能存在误差。事实请以原文为准。

正文节选

The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning Abstract Reasoning models do not stop when they know the answer. On DeepSeek-R1-Distill-Qwen-7B the chain of thought runs about twice as long as the model’s own answer probability takes to settle, and how much of that excess is removable varies from problem to problem, so a global length penalty cannot take it out. We take it out by internalizing a causal interpretability finding into the weights. The mechanis


发布时间:2026-09-01 12:00
抓取时间:2026-09-01 12:13
来源机构:arXiv
阅读原文arxiv.org