CLR-Bench:用隐藏规则课程评测自进化智能体
原标题:More Rules, Longer Lists? Benchmarking Self-Evolving Agents on a Hidden-Rule Curriculum
AI 摘要
研究者提出 CLR-Bench 基准,用隐藏规则课程测试自进化智能体能否从稀疏反馈中归纳出未被告知的规则并泛化到未见样本。基准包含 10 个领域 20 条链,每级叠加一条隐藏规则,规则对应公开网络上的命名分组(如航空联盟创始成员)。评估六个自进化方法发现,在认证模型下它们几乎学不到规则,最佳仅比冻结智能体高 5 分;而给出命名分组约定的固定提示可提升 14.5-16.5 分,使用更强 GPT-6-Sol 时 ACE 和 Dynamic Cheatsheet 可达 78% 准确率。
正文节选
More Rules, Longer Lists? Benchmarking Self-Evolving Agents on a Hidden-Rule Curriculum Abstract. Self-evolving agents rewrite a persistent state from feedback, but it is largely untested whether this lets them learn a rule nobody told them. An agent can also pass by writing down each correction as an exception, and its list then grows with every new rule. We introduce CLR-Bench, a Curriculum of Latent Rules benchmark. Each chain adds one hidden rule per level on top of the earlier ones. A rule