EPD 分离技术加速多模态模型推理的适用场景
原标题：When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving
AI 摘要
NVIDIA 技术博客介绍了编码-预填充-解码（EPD）分离技术，用于加速多模态模型推理。该技术将视觉编码阶段与 LLM 预填充和解码阶段分离，通过 NVIDIA Dynamo 框架实现独立扩展，在图像密集型提示和短输出场景下，可将首令牌时间（TTFT）提升至 5 倍，端到端响应时间提升至 7 倍。文章还讨论了 EPD 的适用场景和硬件放置选项，并指出在长输出或大型密集模型下收益有限。
正文节选
Encode-prefill-decode (EPD) disaggregation is an inference optimization technique for multimodal models that separates the vision encoder stage from the prefill and decode stages. It is most effective for image-heavy prompts, short-to-medium outputs, and quantized mixture-of-experts (MoE) models. This post shows when and how to use EPD disaggregation with NVIDIA Dynamo to achieve up to 5x faster time to first token (TTFT) and 7x faster end-to-end response time. The post also addresses the scenar