GGUF 量化的逐张量布局映射
原标题：Per-tensor layout maps for GGUF quantization
AI 摘要
Hugging Face 博客介绍了一种针对 GGUF 量化的逐张量布局映射方法。作者利用 Claude Code 在 Framework 提供的 AMD AI Max+ 395 128GB 桌面上运行约 96 小时，对 Qwen3.5-0.8B 和 Qwen3.5-4B 等模型进行了 1000 多次量化实验，通过 degrade-one 和 upgrade-one 扫描测量各张量的敏感度。研究发现 token_embd 敏感度最高、深度呈 U 形敏感度分布、小型注意力投影每比特最敏感，并据此构建求解器以生成更优的量化布局。
正文节选
A couple years ago, around the time MoE models started getting more mainstream, many found that the existing algorithm wasn't doing a good job handling them. The introduction of more than 8 experts (llama-quant.cpp only has special cases for mixtral at exactly 8 experts), and the realization that the shexp tensors were pretty sensitive despite being incredibly small, led to a need for extra work to come in. You may know some of these methods, and I maintain a copy of llama-quant.cpp with my own