FlexFloor：弹性专家并行实现资源高效的强化学习后训练
原标题：Resource-Efficient Reinforcement Learning Post-Training through Elastic Expert Parallelism
AI 摘要
清华大学的研究者提出 FlexFloor 系统，用于解决 MoE 模型强化学习后训练中响应生成阶段的资源浪费问题：部分 worker 完成自身响应后，因其他 worker 仍在处理长响应而必须继续占用专家资源。FlexFloor 通过联合调整响应调度与活跃 worker 组，在保持专家可用性的前提下逐步释放已完成 worker。相比 verl 基线，其单位占用计算时间的响应 token 生成量提升约 65.40%，吞吐提升 52.74%，且 rollout 奖励基本不变。
正文节选
marginparsep has been altered. topmargin has been altered. marginparwidth has been altered. marginparpush has been altered. The page layout violates the ICML style. Please do not change the page layout, or include packages like geometry, savetrees, or fullpage, which change it for you. We’re not able to reliably undo arbitrary changes to the style. Please remove the offending package(s), or layout-changing commands and try again. Resource-Efficient Reinforcement Learning Post-Training through El