Learn2Play Bench：LLM 智能体在陌生环境中从经验学习的能力评估
原标题：Paper page - Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?
AI 摘要
研究者提出 Learn2Play Bench，一个由规则新颖或反直觉的文本游戏组成的基准，用于评估 LLM 智能体在陌生环境中通过交互学习的能力，避免依赖预训练知识或指令中给出的规则。该基准提供可复现反馈与自动评分，并变化游戏实例以测试知识迁移。实验评估了骨干模型、自演化方法和智能体框架对学习能力的影响，得出三条发现：保留完整动作与反馈记录优于总结成规则；顶尖人类玩家峰值分数高于被测智能体；固定骨干模型时更换框架可提升性能并降低推理成本。
正文节选
Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? Abstract Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments. Evaluating this ability is therefore important for understanding how effectively agents acquire and use new knowledge. Existing benchmarks have sought to evaluate this ability, but they primarily evaluate tasks whose rules are provided in the instructions or already familiar to pretrained models,