置信度门控分层LLM推理的可行性条件
原标题：A Feasibility Condition for Confidence-Gated Tiered LLM Inference
AI 摘要
该论文研究分层LLM推理中基于置信度信号的路由级联，将路由决策建模为二分类检验，并引入架构成本比κ（本地前向传播成本与升级成本之比），推导出一个闭式可行性条件：当该条件不满足时，任何阈值都无法使级联优于将所有查询发送到后端。作者指出，在本地计算需要计费时成本约束起主导作用，而在本地计算为沉没成本（如去中心化硬件）时可行性变为无条件，信号质量转而决定级联效果。该工作将条件转化为三项设计要求，并提出未来协同设计架构与协议。
正文节选
A Feasibility Condition for Confidence-Gated Tiered LLM Inference Abstract Tiered inference distributes large language model computation across a hierarchy of devices, from on-device and edge hardware to the datacenter, and is increasingly offered as the route to cheaper, lower-latency, and more private serving. Most such systems orchestrate frozen model checkpoints, and most decide what to resolve locally and what to escalate using a confidence signal produced by a forward-deployed model. We gi