预测性人类偏好：从模型排名到模型路由
原标题：Predictive Human Preference: From Model Ranking to Model Routing
AI 摘要
Chip Huyen 在博客中探讨了预测性人类偏好（Predictive Human Preference）的概念，旨在预测用户对特定提示词更偏好哪个模型，并应用于模型路由和可解释性。她使用 LMSYS Chatbot Arena 的公开数据（2023年7月，33K 比较）作为基准，评估了 Bradley-Terry 等排名算法的准确性，并构建了一个玩具偏好预测器来预测模型对之间的胜负。文章还讨论了 Chatbot Arena 排名算法的正确性，指出其实际使用 Bradley-Terry 而非 Elo，并提到生产环境中用户投票噪声的问题。
正文节选
A challenge of building AI applications is choosing which model to use. What if we don’t have to? What if we can predict the best model for any prompt? Predictive human preference aims to predict which model users might prefer for a specific query. Human preference has emerged to be both the Northstar and a powerful tool for AI model development. Human preference guides post-training techniques including RLHF and DPO. Human preference is also used to rank AI models, as used by LMSYS’s Chatbot Ar