llama.cpp b11476:Metal 新增多类型 few-row MMA 矩阵乘法内核
原标题:b11476
AI 摘要
llama.cpp 发布 b11476 版本,为 Metal 后端新增 few-row MMA 矩阵乘法内核,覆盖 BF16、Q1_0、Q2_0、MXFP4、Q2_K、Q3_K、TQ2_0 及 IQ 系列等剩余 src0 类型。该内核在 M3 Ultra 上按类型设定生效行数阈值(TQ2_0 为 5 行,BF16 为 4 行,MXFP4/Q2_0/Q2_K/IQ4_NL 为 3 行,其余为 2 行)。在 m=4096、k=14336 的 MUL_MAT 基准测试中,阈值至 8 行区间性能比 master 提升约 0.23 至 0.98 倍。
正文节选
<details open> metal : few-row MMA mat-mul for the remaining src0 types (#30065) The generic few-row MMA kernel works for any type with a 16-weight dequantizer, so it now also takes BF16, Q1_0, Q2_0, MXFP4, Q2_K, Q3_K, TQ2_0 and the IQ types. Each type starts at the row count where it beats the current kernels on an M3 Ultra: 5 rows for TQ2_0, 4 for BF16, 3 for MXFP4, Q2_0, Q2_K and IQ4_NL, and 2 for the others. test-backend-ops perf -o MUL_MAT, m=4096, k=14336, M3 Ultra, time of this change