SilentProbe：测量生产 API 作为代理工具时的静默失败
原标题：SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools
AI 摘要
SilentProbe 研究测量了生产 API 作为 LLM 代理工具时的静默失败问题。通过审计 2501 个 OpenAPI 文档和 721320 个参数，发现 40.1% 的文档在 schema 中未编码约束，导致代理无法区分空结果与未理解的查询。实验表明，机器可检查的约束在 111 次中全部返回诚实错误，而纯文本约束在 61 次中有 44 次静默失败。将词汇表提升到 schema 中可将失败率从 88/88 降至 0/89，表明修复一行 schema 比改进模型更有效。
正文节选
SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools Abstract. An LLM agent calling a production API cannot distinguish a query that matched nothing from a query the server did not understand. Both return HTTP 200 with a parsable body, no exception to catch and no field to branch on. We ask what predicts which one occurred, and what it does to the agent. Auditing 721,320 parameters across 2,501 independently published OpenAPI documents, we find that 7.5% declare an enumer