LLM推理网络的最优模型激活策略
原标题：Optimal Model Activation Policies for Inference Networks of Large Language Models
AI 摘要
该论文提出「推理网络」（inference networks）框架，将多个LLM专家建模为有向图，节点为模型、边表示条件激活，以在给定性能约束下最小化推理成本。作者针对串联拓扑证明最优激活策略具有阈值结构：先调用最低成本模型，仅当置信度低于阈值时才调用更贵模型；判别任务每类一个阈值，生成任务仅一个阈值。实验在开源LLM的分类与生成基准上显示，在满足性能预算的同时显著降低成本。
正文节选
Optimal Model Activation Policies for Inference Networks of Large Language Models Abstract Recent advances in large language models (LLMs) have rendered them necessary for Natural Language Processing (NLP) tasks, and their high inference cost motivates the study of cost–performance trade-offs to carry out these tasks. In practice, several expert LLMs are used in synergy for inference, either in an ensemble mode or in series, yet without a principled approach on how to best use the available mod