Amazon SageMaker AI 上的 MoE kernel 工程优化
原标题：Mixture-of-Experts kernel engineering on Amazon SageMaker AI
AI 摘要
AWS Machine Learning Blog 发布技术文章，介绍在 Amazon SageMaker AI 上针对 Mixture-of-Experts（MoE）模型的 kernel 工程优化。文章重点讲解 Fused MoE 技术，以及如何通过 Swap-AB 转置矩阵乘法来适配 NVIDIA Hopper 架构的 WGMMA 指令，解决 decode 阶段 M 维度过小导致的 Tensor Core 利用率低下问题。该优化可提升 MoE 推理的 GPU 利用率和性能。
正文节选
Mixture-of-Experts kernel engineering on Amazon SageMaker AI Mixture-of-Experts (MoE) is a sparsely-activated architecture that scales large language model (LLM) capacity while keeping per-token compute low. Because its operations account for a large share of total compute time, inefficiencies there directly raise request latency and cost. Therefore, optimizing MoE operations through kernel engineering offers the largest opportunity for performance gains. Fused MoE is a key optimization techniqu