Together AI 推出专用模型推理，实现容量感知路由

AI 摘要

Together AI 平台推出了专用模型推理功能，由端点、部署和配置三部分组成，并通过基于容量的流量分配实现路由。该架构支持 A/B 测试、影子实验和零停机变更，且配置不可变，确保行为稳定。文章详细解释了权重与副本数的关系，以及如何通过配置选择优化延迟或吞吐量。

正文节选

Dedicated Model Inference on the Together AI platform consists of three parts: the endpoint (a stable name you or your clients call), deployments (specific model + hardware combinations running replicas behind it), and configs (recipes for how a model runs). A capacity-aware traffic split ties these three entities together. This architecture enables various other features such as rollouts, A/B tests, shadow experiments, zero-downtime changes possible. Below we'll show how capacity-aware routing

发布时间：—

抓取时间：2026-08-03 01:12

来源机构：Together AI