GradCuit:通过信用分配梯度流实现稳健可解释的测试时潜在推理
原标题:GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
AI 摘要
GradCuit是一种新的测试时潜在推理方法,通过在选定的Transformer层插入可优化的潜在状态,利用因果自注意力机制实现直接且可解释的梯度分配。在五个指令微调骨干模型、三个推理基准和两种答案格式上,GradCuit平均准确率达64.5%,优于链式思维提示6.6个百分点,并展现出更强的鲁棒性。该方法还提供了token级梯度归因,揭示潜在影响集中在推理连接token上,并识别早期到中层Transformer层为最有效的优化空间。
正文节选
GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning Abstract Optimization-based latent reasoning improves large language model outputs by optimizing instance-specific continuous states at test time while keeping model parameters frozen. Existing methods, however, typically connect these states to the reasoning trajectory through decoded tokens, making sequence-level credit assignment indirect and obscuring how latent updates shape subsequent reason