Netflix 提出推荐系统反事实可观测性评估框架
原标题：Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix
AI 摘要
Netflix 在 arXiv 发表论文，提出一套面向推荐系统的反事实可观测性评估框架，用于区分内容质量与模型行为。该框架以带探索日志的选择概率为基础，提供偏差削减、相对性和增量性三类测量方法，覆盖单阶段与级联推荐系统。论文称该框架已在 Netflix 多个生产系统部署，并通过模拟、与历史 A/B 测试对齐及一个发现标签归因错误的生产监控案例得到验证。
正文节选
Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix Abstract. Understanding the performance of large-scale recommender systems remains an underexplored challenge, especially for content creators and model developers. The raw engagement signals available to them, such as views and clicks, conflate content quality, model behavior, presentation bias, and audience reach, making it hard to attribute outcomes to the right cause. In this work, we present a