FocusGUI：面向全模态 GUI 智能体的聚焦感知轻量适配器
原标题：Paper page - Focused Omni-Modal GUI Agents
AI 摘要
该论文提出 FocusGUI，一种轻量级适配器，用于 GUI 智能体在多模态流式观测（截图、音频、视频、交互历史、设备状态）中动态筛选与当前步骤相关的证据，避免无关上下文导致误差累积和轨迹漂移。FocusGUI 仅用动作监督训练，无需模态相关性标签，也不微调骨干模型，而是将压缩后的相关证据注入冻结解码器的嵌入中。在四个全模态骨干上，其 Type Match 平均提升 33.2，Exact Match 提升 3.0 倍，并比最强现有增强方法高 30.1。
正文节选
Abstract FocusGUI is a lightweight adapter that dynamically selects relevant multimodal evidence for GUI agents to prevent compounding errors from irrelevant context, improving action accuracy without fine-tuning the backbone. Graphical User Interface (GUI) agents increasingly act over streaming omni-modal observations spanning screenshots, audio, video, interaction history, and device states. However, exposing this complete context uniformly at every step ignores that relevance is transient and