Octopus：面向潜在 KV 多智能体强化学习的分支执行框架
原标题：Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning
AI 摘要
该论文提出 Octopus，一个将服务运行时、GPU 内存管理与强化学习框架协同设计的系统，把智能体执行过程建模为 GPU 常驻的 DAG，通过 fork、rewind、read_thread、wait、done 五个原语在不可变 radix 游标上协调多智能体。它仅在 full-attention 层拼接兄弟 KV 页、循环层跳过挂载区间，将离任务劫持从 57.9% 降至 2.7%，并配合 DAG-GRPO 目标将 rollout 与训练的 KL 散度降低 32 倍。基于 Qwen3.5-4B 训练后，Octopus 在 BrowseComp-Plus 稀疏 BM25 下准确率从 26.0% 提升至 49.6%，超过多个更大规模模型。
正文节选
Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning Abstract Deep search and long-horizon reasoning naturally unfold as branching trajectories that pursue parallel sub-queries, abandon dead-end detours, and cross-check sibling findings. Yet modern serving engines, agent harnesses, and reinforcement learning (RL) trainers execute every episode as a flat, append-only text sequence: stretching a single context window leaves failed detours in working memory that di