Wieszcz-XIX：31亿词1918年前波兰语语料库及时间受限语言模型
原标题：Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch
AI 摘要
研究者构建了 Wieszcz-XIX，一个包含 67.5 亿 token（约 31 亿词）、294,369 篇文档的 1800-1918 年波兰语历史语料库，规模比同期人工标注语料大三个数量级以上，并公开了构建流程、代码与权重。基于该语料，他们从零训练了 47M 到 349M 参数的仅解码器模型阶梯，测量其时间边界性：349M 和 107M 模型在时期词汇上比现代波兰语基线模型每字节少用约 3.1 比特，而在 1918 年后词汇上多用约 3.1 比特，显示出交叉效应。研究还量化了 OCR 损坏、重复和 1918 年后数据泄漏等语料缺陷，并警告模型会复现包括反犹言论在内的时代偏见。
正文节选
Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch Abstract Historical Polish is well documented as a language but annotated in machine-readable form only to about a million words for the period this paper covers; the rest sits behind optical character recognition of variable quality. We present Wieszcz-XIX, a corpus of 6.75 billion tokens (about 3.1 billion words) in 294,369 documents, most of them periodical issues, of Polish p