DivAlgo：代码生成模型算法多样性系统评估
原标题：DivAlgo: A systematic evaluation of algorithmic diversity for code generation models
AI 摘要
研究者提出 DivAlgo 基准，用于系统评估代码生成模型的算法多样性，包含算法上不同的编程题解数据集和轻量级相似度指标。对 24 个现代代码生成模型的实验显示，模型生成的代码多样性仅为人类解法的 31%，即使采用温度变化、verbalized sampling 等多样性策略也只能达到 70.5%。研究还发现模型倾向于对齐更流行的人类解法，且无法复现人类解法的整体多样性，提示过度接触模型生成代码可能收窄程序员的算法词汇。
正文节选
DivAlgo: A systematic evaluation of algorithmic diversity for code generation models Abstract Code generation models are actively being used in professional and educational settings. However, the competence of large language models (LLMs) to generate algorithmically diverse code remains largely unknown. We propose the DivAlgo benchmark, a large-scale dataset constituting algorithmically diverse solutions to programming problems. We also introduce a lightweight similarity metric to enable large-s