纯Rust端到端训练语言模型：经验报告
原标题：Training a Language Model End-to-End in Rust: An Experience Report
AI 摘要
作者独自用纯 Rust 在租用 GPU 上以 164 美元成本端到端预训练了一个约 0.4B 参数、以孟加拉语为主的语言模型，并记录了 Candle 与 Burn 两个 Rust ML 框架作为训练后端时的缺陷分类：Candle 有 5 个缺陷（包括融合核静默不产生梯度），Burn 有 3 个缺陷（包括反向传播吞吐仅约理论值 3%、核融合路径在数十亿参数规模下训练中途段错误）。作者提出以梯度流仲裁器为核心的验证纪律，在浪费算力前捕获了六类静默失败，并发现孟加拉文字节级分词导致约 1.4 字符/词元、修复后约 4.1 字符/词元。最终作者将训练迁回 PyTorch，仅保留 Rust 用于端侧推理，认为当前 Rust 尚不适合训练语言模型但适合服务模型。
正文节选
Training a Language Model End-to-End in Rust: An Experience Report Abstract I pretrained a language model end-to-end in Rust — alone, with no team, no PyTorch, and no Python anywhere in the training path — for $164 in rented GPU time. I report that as an achievement, not a recommendation: the more useful contribution of this paper is a measured failure taxonomy of the two leading Rust ML frameworks, Candle and Burn, as training (not inference) backends in 2026 — five distinct Candle defects, inc