Token 膨胀感知路由：面向智能体 LLM 系统的成本优化
原标题：Not All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM Systems
AI 摘要
arXiv 论文提出 InflationAgent，一种面向智能体 LLM 系统的路由框架，通过量化 token 膨胀（重试导致的真实成本与单次调用成本之比）来优化模型选择。其引入 CoT 分支熵（CBE）作为预执行难度信号，并以语义汇率（SER）为目标进行路由，在 GSM8K 上以更少 token 达到更高准确率，同时验证了失败链转发会显著降低强模型性能。
正文节选
Not All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM Systems Abstract When a language model fails to answer a query on the first attempt, an agentic system retries, consuming additional tokens each time. This retry overhead creates a gap between what a model’s per-token price implies and what a full workflow actually costs. We call this gap token inflation and define it as the ratio of true workflow cost to single-call cost. Systems like FrugalGPT [1] route based on the latter, whic