学习式上下文规划未能超越强检索:长上下文问答的受控研究
原标题:When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA
AI 摘要
该研究在 LongBench-v2 多选题问答上系统测试了学习式上下文规划(learned context planning)能否在强检索、路由、预算化选择和重排序控制下提升长上下文问答表现。使用 Qwen2.5-7B-Instruct 作为答案模型,规划器通过 SFT 在 140 条训练和 28 条开发轨迹上训练。结果显示,在 18k 字符预算下,锚定混合检索达到 36.18% 准确率,BM25 为 35.98%,而最佳规划器引导方法仅 34.19%;在未触碰的 152 题测试集上,锚定混合检索仍更高(42.11% vs 36.84%)。作者结论是学习式规划在此设置下只是弱相关性信号,不能替代强检索。
正文节选
When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA Abstract Learned context planning selects evidence atoms before an answer model reasons over them. We test whether this learned selection improves long-context multiple-choice QA after strong retrieval, routing, budgeted-selector, and reranking controls. Our primary diagnostic uses all 503 LongBench-v2 MCQ questions with Qwen2.5-7B-Instruct. The planner is SFT-