Elastic 分享为智能体 AI 产品构建可复用评估框架
原标题:Presentation: Building Reusable Evaluation Frameworks for Agentic AI Products
AI 摘要
Elastic 首席数据科学家 Susan Chang 在演讲中介绍了公司为生产环境中的 AI 智能体构建可复用评估框架的实践。Elastic 内部运行攻击发现(attack discovery)和企业聊天机器人等智能体,各团队原本各自创建数据集、评估器和追踪存储,导致重复工作。公司正从分散评估转向共享框架,并强调追踪(tracing)是评估的基础。
正文节选
Transcript Susan Chang: My name is Susan. I'm a principal data scientist at Elastic. We make tools like Elasticsearch and Kibana. I've also written a book with O'Reilly called "Machine Learning Interviews." Today we're going to be talking about a few things. I'm going to talk a bit about what kind of agents we've been building and try to maybe connect them to what you folks might have been building. Tracing, which is, in my opinion, very important and a huge foundation, as well as how we use tha