GPT-SW3: An Autoregressive Language Model for the Nordic Languages
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910661620006912 |
|---|---|
| author | Ekgren, Ariel Gyllensten, Amaru Cuba Stollenwerk, Felix Öhman, Joey Isbister, Tim Gogoulou, Evangelia Carlsson, Fredrik Heiman, Alice Casademont, Judit Sahlgren, Magnus |
| author_facet | Ekgren, Ariel Gyllensten, Amaru Cuba Stollenwerk, Felix Öhman, Joey Isbister, Tim Gogoulou, Evangelia Carlsson, Fredrik Heiman, Alice Casademont, Judit Sahlgren, Magnus |
| contents | This paper details the process of developing the first native large generative language model for the Nordic languages, GPT-SW3. We cover all parts of the development process, from data collection and processing, training configuration and instruction finetuning, to evaluation and considerations for release strategies. We hope that this paper can serve as a guide and reference for other researchers that undertake the development of large generative models for smaller languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_12987 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | GPT-SW3: An Autoregressive Language Model for the Nordic Languages Ekgren, Ariel Gyllensten, Amaru Cuba Stollenwerk, Felix Öhman, Joey Isbister, Tim Gogoulou, Evangelia Carlsson, Fredrik Heiman, Alice Casademont, Judit Sahlgren, Magnus Computation and Language Artificial Intelligence This paper details the process of developing the first native large generative language model for the Nordic languages, GPT-SW3. We cover all parts of the development process, from data collection and processing, training configuration and instruction finetuning, to evaluation and considerations for release strategies. We hope that this paper can serve as a guide and reference for other researchers that undertake the development of large generative models for smaller languages. |
| title | GPT-SW3: An Autoregressive Language Model for the Nordic Languages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2305.12987 |