Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.00920 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908931094216704 |
|---|---|
| author | van Oort, Jesse Brinkkemper, Frank de Graaf, Erik Vanroy, Bram Lensink, Saskia |
| author_facet | van Oort, Jesse Brinkkemper, Frank de Graaf, Erik Vanroy, Bram Lensink, Saskia |
| contents | We present the GPT-NL Public Corpus, the biggest permissively licensed corpus of Dutch language resources. The GPT-NL Public Corpus contains 21 Dutch-only collections totalling 36B preprocessed Dutch tokens not present in any other LLM pretraining corpus. Additionally, the corpus includes roughly 207B English, 232B Code, and 48B German/Danish tokens taken from existing sets which we further curated for compliance. This corpus includes curated data from large existing corpora like Common Corpus and Common Crawl, as well as newly created Dutch-specific collections. Most newly created Dutch collections consist of content collected in collaboration with organisations or synthetically augmented content. All data is collected and evaluated with the aim of facilitating the creation of (commercial) language models that are lawful, useful and non-harmful. All data included in the GPT-NL Public Corpus is sourced from datasets with permissive licensing and is curated and redistributed under a CC-BY license. The full dataset is publicly available on the Hugging Face Hub. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_00920 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GPT-NL Public Corpus: A Permissively Licensed, Dutch-First Dataset for LLM Pre-training van Oort, Jesse Brinkkemper, Frank de Graaf, Erik Vanroy, Bram Lensink, Saskia Computation and Language We present the GPT-NL Public Corpus, the biggest permissively licensed corpus of Dutch language resources. The GPT-NL Public Corpus contains 21 Dutch-only collections totalling 36B preprocessed Dutch tokens not present in any other LLM pretraining corpus. Additionally, the corpus includes roughly 207B English, 232B Code, and 48B German/Danish tokens taken from existing sets which we further curated for compliance. This corpus includes curated data from large existing corpora like Common Corpus and Common Crawl, as well as newly created Dutch-specific collections. Most newly created Dutch collections consist of content collected in collaboration with organisations or synthetically augmented content. All data is collected and evaluated with the aim of facilitating the creation of (commercial) language models that are lawful, useful and non-harmful. All data included in the GPT-NL Public Corpus is sourced from datasets with permissive licensing and is curated and redistributed under a CC-BY license. The full dataset is publicly available on the Hugging Face Hub. |
| title | GPT-NL Public Corpus: A Permissively Licensed, Dutch-First Dataset for LLM Pre-training |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.00920 |