ThunderKittens 针对 NVIDIA Blackwell GPU 优化并开源新内核
原标题：ThunderKittens Now Optimized for NVIDIA Blackwell GPUs
AI 摘要
Together AI 与斯坦福大学合作开发了 ThunderKittens 框架，并针对 NVIDIA Blackwell 架构开源了新的内核，包括 BF16/FP8 GEMM 和注意力前向/反向内核。这些内核在 B200 上接近 cuBLAS/cuDNN 速度，在 H100 上比 cuBLAS 和 FA3 快达 2 倍。文章深入探讨了如何利用第五代张量核心、张量内存和 CTA 对等新特性，通过数据流优化实现高性能。
正文节选
At Together AI, we have been investing in the ThunderKittens framework - a software library that we developed in collaboration with researchers at Stanford to make it easier to quickly write performant GPU kernels. Today, we're open-sourcing some kernels written for NVIDIA Blackwell architecture. Our customers can use these on Together GPU Clusters today, and they're coming soon to the Together Kernel Collection. This blog is cross-posted to the Hazy Research blog at Stanford University. We've b