Together AI 发布 QTIP：新型权重量化方法实现质量与速度兼得
原标题：We’re excited to announce QTIP (Quantization with Trellises and Incoherence Processing), a new weight-only LLM post-training quantization method that achieves a state-of-the-art combination of quality
AI 摘要
Together AI 发布了 QTIP，一种新的仅权重量化方法，结合 Trellis 编码量化与不连贯处理，在模型质量和推理速度上达到最先进水平。QTIP 在 QuIP# 基础上改进，速度比未量化模型快 3 倍以上，并已发布预量化模型（如 Llama 3.1 405B Instruct）和代码库。该论文将作为 Spotlight 出现在 NeurIPS 2024。
正文节选
We’re excited to announce QTIP (Quantization with Trellises and Incoherence Processing), a new weight-only LLM post-training quantization method that achieves a state-of-the-art combination of quality and inference speed! QTIP compresses the weights of a model with trellis coded quantization, which achieves near-optimal distortion on a wide range of distributions. QTIP significantly improves over QuIP#’s [1] quality while being >3X faster than unquantized models. How Does QTIP Work? QTIP has two