遗憾主导意外：智能体AI安全的设计时需求工程
原标题：Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI Safety
AI 摘要
该研究提出一种基于目标导向需求工程（GORE）的MS-RGR机制，用于在智能体AI系统设计阶段建模和模拟安全自主性，通过结合认知遗憾和认知意外两种信号，解决安全自主性的三难问题（常规自主、反思推理或升级至人类）。在老年护理和自动驾驶两个工作流领域的随机模拟中，MS-RGR将静默失败率降至接近零，风险检测速度比传感器基线快约17.5倍，并通过LTL安全属性保持可追溯性。在AgentHarm基准的208个场景中，MS-RGR仅对两个基线安全行为较强的模型显著提升有害任务拒绝率（如84.1%→90.9%），表明其增强而非替代模型级安全训练。
正文节选
Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI Safety Abstract Requirements Engineers for Agentic-AI domains face challenges in evaluating and verifying the modeling, elaboration, and operationalization of safe autonomy in these systems. Mainstream frameworks, such as Goal-Oriented Requirements Engineering (gore), lack mechanisms to systematically address these challenges in the face of epistemic uncertainty. We contribute to an approach that builds on gore to mod