JAVIS：面向实时对话智能体的联合音视频交互与流式生成
原标题：JAVIS: Joint Audio-Visual Interaction and Streaming for Real-Time Conversational Agents
AI 摘要
韩国中央大学的研究者提出 JAVIS，一个端到端流式音视频对话智能体框架，让语音和面部表情从单一共享表示中联合生成，避免传统级联管线中的信息瓶颈与延迟累积。该方法基于预训练 Fun-Audio-Chat，通过 LoRA 将语音精炼头改造为联合精炼头，输出统一音视频表示并分送语音与面部动作两个流式分支。团队还构建了 334 小时的合成音视频对话语料 JAVIS-Syn，实验显示其初始延迟比强基线低约 200，并在唇同步与表情适切性上优于基线。
正文节选
JAVIS: Joint Audio-Visual Interaction and Streaming for Real-Time Conversational Agents Abstract Human communication is inherently multimodal, conveying intent not only through linguistic and vocal cues but also through visual signals. However, existing conversational systems have mainly focused on text or speech, leaving out the visual feedback that is essential for natural interaction. In this paper, we propose JAVIS, an end-to-end streaming framework for real-time audio-visual conversational