CascadeRec：多租户内容推荐的自适应 LLM 算力分配
原标题：CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation
AI 摘要
Workday AI Research 提出并部署了 CascadeRec，一个面向多租户 FP&A SaaS 内容推荐的两级 LLM 重排序级联系统：轻量 Tier-1 模型处理所有请求，路由器仅将可能受益的请求升级到更昂贵的 Tier-2 流程。该系统在生产 A/B 实验中运行超过一个月，推理成本降低约 10%，延迟影响极小且未观察到质量回退。论文还基于公开基准发现，统一使用 LLM 重排序会使稀疏历史用户的 NDCG@10 最多下降 30%，而对这类用户升级计算反而可能扩大损失。
正文节选
CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation Abstract Large language models (LLMs) are attractive rerankers for recommendation. Running an expensive model on every request is costly, and for users with sparse interaction histories it often lowers quality. We present CascadeRec, a two-tier LLM reranking cascade deployed in production at a large financial planning and analysis (FP&A) SaaS provider. It serves content recommendations to enterprise tena