MedJev：评估医学零样本System-One决策
原标题：MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine
AI 摘要
研究团队发布开源 System-One 模型 MedJev，可在笔记本本地运行，在十个医学基准上评估零样本决策能力。在 MedQA、USMLE、NEJM 上分别达到 72.5%、75.4%、80.1% 准确率，与已发表的零样本 GPT-4 结果相差 2-4 个百分点，并显著超过 GPT-3.5。在 MIMIC-IV 院内死亡预测上 AUROC 达 0.926，匹配或超过监督模型 XGBoost（0.902），但急诊分诊等运营类任务弱于监督模型，且患者级概率校准受本地患病率影响（ECE 0.286）。作者提出两参数轻量重校准方法，用 50-200 个标注样本将 ECE 降至 0.007，并开源代码与评测框架。
正文节选
MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine Abstract System-One decision models such as Jev answer structured questions about a given state, including option selection and probability estimation, in a single forward pass without generating text. Their speed, low computational cost, and ability to produce calibrated probabilities make them promising for clinical decision support. We investigate the zero-shot performance of MedJev, an open System-One model that runs locally