DevIntent：LLM 生成代码违背开发者意图的程度研究

AI 摘要

arXiv 论文提出 DevIntent 基准，通过意图违反率（IVR）衡量 LLM 生成代码是否违背开发者隐含意图。在 49 个问题上，Claude Sonnet 4.6 和 GPT 4.1 虽通过 92% 的显式测试，但分别有 54.5% 和 63.5% 的问题违反隐含约束，表明通过率高估了代码与开发者意图的一致性。

正文节选

Computer Science > Software Engineering Title:DevIntent: How Much Does LLM-Generated Code Violate Developer Intent? View PDF HTML (experimental) Abstract:Code generated by LLMs can violate a developer's implicit intentions when given an ambiguous prompt, yet standard benchmarks measure only whether code passes its stated test. We introduce the Intent Violation Rate (IVR) and a 49-problem pilot benchmark derived from HumanEval+. Each problem strips implicit constraints from a clarifie

发布时间：2026-08-11 12:00

抓取时间：2026-08-11 13:02

来源机构：arXiv