Google DeepMind 发布 T5Gemma：基于适配技术的编码器-解码器模型系列
原标题：T5Gemma: A new collection of encoder-decoder Gemma models
AI 摘要
Google DeepMind 发布了 T5Gemma，这是一系列通过模型适配技术将预训练的仅解码器模型转换为编码器-解码器架构的模型，基于 Gemma 2 框架，包括适配的 2B 和 9B 模型以及新训练的 T5 规模模型。实验表明，T5Gemma 在多个基准上达到或超过仅解码器对应模型的性能，并在推理效率上具有优势，例如 T5Gemma 9B-2B 在 GSM8K 上精度提升且延迟接近 2B 模型。该系列模型已向社区发布，以促进研究。
正文节选
In the rapidly evolving landscape of large language models (LLMs), the spotlight has largely focused on the decoder-only architecture. While these models have shown impressive capabilities across a wide range of generation tasks, the classic encoder-decoder architecture, such as T5 (The Text-to-Text Transfer Transformer), remains a popular choice for many real-world applications. Encoder-decoder models often excel at summarization, translation, QA, and more due to their high inference efficiency