TraceDance：从真实部署轨迹自动构建 Agent 行为基准
原标题：Paper page - TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces
AI 摘要
研究者提出 TraceDance，一个能从真实 Agent 部署轨迹中自动构建针对性行为基准的 Agent 系统，用于检测用户指定的不良行为。它通过 Anchor-and-Confirm 检索与 Flash LLM 确认、Anchor Synthesis Loop 生成行为规范，并采用决策点续写方式评估 LLM 的下一轮响应。实验基于 252,557 个会话生成 107 个基准、4,125 个实例，满足 95.3% 的构建目标请求；九款前沿 LLM 平均通过率仅 26.7%，显示其在 Agent 决策点仍表现不佳。
正文节选
TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces Abstract An agent can complete a task while exhibiting undesirable behavior during execution. Developers need tests for the specific behaviors encountered in deployment, beyond fixed benchmark suites. We present TraceDance, an agent system that constructs targeted benchmarks from deployment traces for user-specified undesirable behaviors. For efficient construction, Anchor-and-Confirm c