llama.cpp b11095:Hexagon HMX 优化 GATED_DELTA_NET
原标题:b11095
AI 摘要
llama.cpp 发布 b11095 版本,主要新增针对高通 Hexagon 平台的 HMX 优化版 GATED_DELTA_NET(GDN)内核。该 PR 逐步实现了 HMX 支持、流水线化 HMX 与 DMA、HVX 多线程、向量化 expf 及尾部处理优化,并改进了 flash attention 内联 softmax 与 DMA 流水线,使 token 生成期间 DDR 读取减少 20-30%。同时附带 macOS、Linux、Windows、Android 等多平台预编译二进制包。
正文节选
<details open> hexagon: new HMX-optimized GATED_DELTA_NET (#29199) * hex-gdn: start putting together HMX support for GDN * hex-gdn: working hmx but not-pipelined and slow for now * hex-gdn: re-write vtcm layout handling and prep for pipelining * hex-gdn: starting to pipeline hmx and dmas * hex-gdn: add hvx threading for most pipeline stages * hex-gdb: add detailed trace events * hex-gdn: vectorize expfs and use aligned hvx reads/writes * hex-gnd: vectorize the rest of expf * hex-gdn: o