OpenESM：能量引导模型可作为可扩展的基础学习器

AI 摘要

研究团队提出能量引导模型（ESM），将能量引导与 token 生成整合到单一架构中，先优化潜状态再生成 token，并在三个大规模语料上预训练。在 26 个下游零样本任务上，ESM 展现出随模型规模和数据增长的良好扩展性，并在多数任务上优于 GPT。团队以 OpenESM 名义开源模型权重、训练配方和基础设施，供社区复现与扩展。

正文节选

OpenESM: Energy-Steered Models Are Scalable Foundation Learners Abstract Energy-based models have shown promising properties for domain-specific tasks but remain challenging in modeling general-purpose scenarios. To bridge this gap, we design energy-steered models (ESMs) for foundational language modeling. ESMs integrate energy steering and token generation within a single architecture to leverage the energy-guided search to refine token generation. Specifically, ESMs first optimize a latent sta

发布时间：2026-10-27 12:00

抓取时间：2026-10-27 12:00

来源机构：arXiv