SimuVerity:面向工程级 Simulink 模型生成的智能体基准
原标题:SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation
AI 摘要
研究者提出 SimuVerity,一个包含 101 个文本到可执行 Simulink 模型生成任务的基准,覆盖十个工程领域,每个任务配有可执行系统画像和四类原生仿真场景。其分层评估器先检查交付、可执行性与工程合格性,再从准确度、输出质量、机理保真、控制与因果完整性、工况鲁棒性和动态响应六个维度打分。对六个智能体系统的评测中最高总分仅 42.86,交付率、可执行率、合格率分别为 94.72%、86.47%、76.90%,且部分高分模型仍存在严重布局混乱,说明结构相似度不能代表工程性能。
正文节选
SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation Abstract Existing Simulink benchmarks mainly evaluate whether generated models compile, execute, or resemble a reference model. These criteria do not establish whether a model satisfies its engineering requirements. We introduce SimuVerity, a benchmark of 101 text-to-executable Simulink model-generation tasks across ten engineering domains. For each task, executable-system profiles ground the engineering specificatio