找到着法不等于赢棋：XiangqiBench 闭环评估 LLM 智能体
原标题：Finding the Move Is Not Winning the Game: XiangqiBench for Closed-Loop Evaluation of LLM Agents
AI 摘要
研究者提出 XiangqiBench，一个可执行的中国象棋基准，用于闭环评估 LLM 智能体：从 119 个有引擎或仅将军搜索支持的战术残局出发，智能体需执红方对抗引擎防守方并完成将死。研究记录了 12 个前沿 LLM 在两种观察协议下的 8,568 条多轮轨迹，发现三种看似胜任的信号均高估闭环成功率：转换差距（26.1% 走对参考首步但仅 13.9% 获胜）、一致性差距（pass@3 达 38.7% 而 pass^3 仅 5.9%）、模拟差距（32.3% 被接受的模拟调用停在非法着法，49.3% 情况下真实防守方回应与模拟不同）。结论是智能体评估应评分闭环结果并同时报告可靠性与覆盖率。
正文节选
Finding the Move Is Not Winning the Game: XiangqiBench for Closed-Loop Evaluation of LLM Agents Abstract Static evaluations credit a language model for naming the right move, but an agent must carry a plan through to a verified outcome while an opponent responds. We introduce XiangqiBench, an executable benchmark that measures this difference in Chinese chess: starting from 119 tactical endgames with forced mates supported by engine or checks-only search, an LLM agent must deliver checkmate agai