GardenBench:轻量级每日 LLM 能力评测基准
原标题:GardenBench: A lightweight, daily evaluation of LLM capabilities
AI 摘要
研究者提出 GardenBench,一个包含 11 个测试套件、105 项测试的轻量级 LLM 基准,通过每日自动运行(GitHub Actions)评估 Anthropic、DeepSeek、OpenAI、Google、Moonshot AI、阿里云和 Cohere 等厂商的前沿模型,记录准确率、成本与耗时的逐日变化。该基准旨在弥补模型卡由开发者自评、一次性发布评估无法捕捉模型漂移(部署后变更或新模型发布导致的性能波动)的不足,并通过公开实时仪表盘向公众开放结果。
正文节选
[3] \KOMAoptioncaptionstableheading GardenBench: A lightweight, daily evaluation of LLM capabilitiesThanks: The live dashboard is available at: https://gardenbench.ai/. We gratefully acknowledge funding from Natural Sciences and Engineering Research Council of Canada (NSERC) Alliance Grant (110_2024_2025_Q4_13) and Social Sciences and Humanities Research Council (SSHRC) Partnership Grant (895-2025-1002). This research was conducted while Benedict Cummins-Mburu, Mariana Garcia Mejia, Oscar Heath,