拒绝不可能：大语言模型代码幻觉的分类法与基准
原标题：Refusing the Impossible: A Taxonomy and Benchmark for Code Hallucination in Large Language Models
AI 摘要
该研究提出了一种代码幻觉的分类法，将代码幻觉定义为无根据的生成，区别于普通代码错误，并构建了包含270个不可满足提示的对抗性基准。对12个开源模型的评估显示，模型在约60%的不可满足提示上生成无根据代码，仅拒绝27%，且未错误拒绝可解控制任务。研究还发现，拒绝行为与表面合理性相关，模型能力提升对理论不可能性幻觉有改善，但对虚构包和API的幻觉影响甚微。
正文节选
Refusing the Impossible: A Taxonomy and Benchmark for Code Hallucination in Large Language Models Abstract Large language models (LLMs) often produce code that looks plausible but is not grounded in reality. The code may import packages that do not exist or claim to implement algorithms that violate proven theorems, while still compiling and running. We study code hallucination as ungrounded generation and separate it from ordinary code error (bugs in otherwise grounded programs). We propose a t