大型语言模型在医学推理中表现出元认知敏感性
原标题：Large Language Models Show Metacognitive Sensitivity in Medical Reasoning
AI 摘要
一项针对医学推理中大型语言模型（LLM）的元认知敏感性的研究，开发了一个受心理物理学启发的临床基准，测试了gpt-4.1-nano在阿尔茨海默型神经认知障碍与抑郁相关认知障碍鉴别中的诊断准确性和置信度行为。结果显示模型表现出部分元认知敏感性，但置信度并非完全可靠，且较新或名义上更强的模型不一定具有更好的置信度-正确性区分能力。研究强调应直接测量医学LLM的置信度，而非仅依赖基准准确率或模型能力推断。
正文节选
Large Language Models Show Metacognitive Sensitivity in Medical Reasoning Abstract Large language models (LLMs) are increasingly evaluated and used in medicine. Clinical usefulness depends not only on answer accuracy, but also on whether confidence tracks evidence quality and uncertainty. Recent work has argued that LLMs lack essential metacognition for reliable medical reasoning, but metacognition can be operationalized in different ways, including missing-answer recognition, knowledge-gap dete