EuroLLM: Multilingual Language Models for Europe
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910618693402624 |
|---|---|
| author | Martins, Pedro Henrique Fernandes, Patrick Alves, João Guerreiro, Nuno M. Rei, Ricardo Alves, Duarte M. Pombal, José Farajian, Amin Faysse, Manuel Klimaszewski, Mateusz Colombo, Pierre Haddow, Barry de Souza, José G. C. Birch, Alexandra Martins, André F. T. |
| author_facet | Martins, Pedro Henrique Fernandes, Patrick Alves, João Guerreiro, Nuno M. Rei, Ricardo Alves, Duarte M. Pombal, José Farajian, Amin Faysse, Manuel Klimaszewski, Mateusz Colombo, Pierre Haddow, Barry de Souza, José G. C. Birch, Alexandra Martins, André F. T. |
| contents | The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date, detailing our data collection and filtering process, the development of scaling laws, the creation of our multilingual tokenizer, and the data mix and modeling configurations. Additionally, we release our initial models: EuroLLM-1.7B and EuroLLM-1.7B-Instruct and report their performance on multilingual general benchmarks and machine translation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16235 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EuroLLM: Multilingual Language Models for Europe Martins, Pedro Henrique Fernandes, Patrick Alves, João Guerreiro, Nuno M. Rei, Ricardo Alves, Duarte M. Pombal, José Farajian, Amin Faysse, Manuel Klimaszewski, Mateusz Colombo, Pierre Haddow, Barry de Souza, José G. C. Birch, Alexandra Martins, André F. T. Computation and Language The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date, detailing our data collection and filtering process, the development of scaling laws, the creation of our multilingual tokenizer, and the data mix and modeling configurations. Additionally, we release our initial models: EuroLLM-1.7B and EuroLLM-1.7B-Instruct and report their performance on multilingual general benchmarks and machine translation. |
| title | EuroLLM: Multilingual Language Models for Europe |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.16235 |