MOSS：搜索与推荐系统语义故障监控诊断框架
原标题：MOSS: A Production Framework for Monitoring and Diagnosing Semantic Failures in Search and Recommendation Systems
AI 摘要
论文提出 MOSS，一个面向搜索与推荐系统的生产级语义质量监控框架，将基于 LLM 的相关性打分嵌入持续监控循环，并加入定向采样、策略感知提示、结构化告警和诊断性追问提示，把低分案例归类为可操作的根因。MOSS 已在美国一家大型零售商的生产环境运行超过六个月，覆盖推荐、搜索和优惠场景，发现两个已确认的服务缺陷，其中一个回填混合缺陷的修复方案经离线评估可使告警率相对下降 76.4%。在 498 对人工标注的推荐基准上，前沿 LLM 评判器的 F1 与 AUC 点估计高于非 LLM 基线，且同一评判器、分数量表和阈值仅需针对场景调整提示即可迁移到 466 对搜索基准。
正文节选
MOSS: A Production Framework for Monitoring and Diagnosing Semantic Failures in Search and Recommendation SystemsThanks: A prior version of this work was presented at the non-archival Unified Search and Recommendation Workshop (USRW), co-located with ACM RecSys 2026. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Abstract Detecting semantic failures in production recommendation