无依赖栈预训练GPT-2 124M并适配Qwen3临床模型
原标题：Pretraining and adapting a language model on a dependency-free stack: GPT-2 124M from random weights, reproduced against llm.c, and a clinical adapter for Qwen3-0.6B
AI 摘要
研究团队使用无第三方运行时依赖的 Zig 机器学习栈 numbat，从随机初始化预训练了 124.4M 参数的 GPT-2，在 9.91B token 网络文本上训练，并与 llm.c 的公开结果对比。其留出交叉熵为 3.2588（参考 3.29），HellaSwag 为 0.3053（参考 0.299），吞吐达 43,374 tokens/秒，高于 PyTorch 的 41,202。团队还将同一栈用于 Qwen3-0.6B 的临床问答适配，留出损失 2.1899。作者强调两个模型均非医疗器械，未经验证用于临床。
正文节选
Pretraining and adapting a language model on a dependency-free stack GPT-2 124M from random weights, reproduced against llm.c, and a clinical adapter for Qwen3-0.6B Abstract Almost every language model in service was trained by one family of software. That concentration makes a question hard to settle: how much of what is known about training a language model describes language models, and how much describes that software? Settling it needs a second implementation able to carry a model through a