Aya 23: Open Weight Releases to Further Multilingual Progress
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914817532493824 |
|---|---|
| author | Aryabumi, Viraat Dang, John Talupuru, Dwarak Dash, Saurabh Cairuz, David Lin, Hangyu Venkitesh, Bharat Smith, Madeline Campos, Jon Ander Tan, Yi Chern Marchisio, Kelly Bartolo, Max Ruder, Sebastian Locatelli, Acyr Kreutzer, Julia Frosst, Nick Gomez, Aidan Blunsom, Phil Fadaee, Marzieh Üstün, Ahmet Hooker, Sara |
| author_facet | Aryabumi, Viraat Dang, John Talupuru, Dwarak Dash, Saurabh Cairuz, David Lin, Hangyu Venkitesh, Bharat Smith, Madeline Campos, Jon Ander Tan, Yi Chern Marchisio, Kelly Bartolo, Max Ruder, Sebastian Locatelli, Acyr Kreutzer, Julia Frosst, Nick Gomez, Aidan Blunsom, Phil Fadaee, Marzieh Üstün, Ahmet Hooker, Sara |
| contents | This technical report introduces Aya 23, a family of multilingual language models. Aya 23 builds on the recent release of the Aya model (Üstün et al., 2024), focusing on pairing a highly performant pre-trained model with the recently released Aya collection (Singh et al., 2024). The result is a powerful multilingual large language model serving 23 languages, expanding state-of-art language modeling capabilities to approximately half of the world's population. The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs breadth, exploring the impact of allocating more capacity to fewer languages that are included during pre-training. Aya 23 outperforms both previous massively multilingual models like Aya 101 for the languages it covers, as well as widely used models like Gemma, Mistral and Mixtral on an extensive range of discriminative and generative tasks. We release the open weights for both the 8B and 35B models as part of our continued commitment for expanding access to multilingual progress. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15032 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Aya 23: Open Weight Releases to Further Multilingual Progress Aryabumi, Viraat Dang, John Talupuru, Dwarak Dash, Saurabh Cairuz, David Lin, Hangyu Venkitesh, Bharat Smith, Madeline Campos, Jon Ander Tan, Yi Chern Marchisio, Kelly Bartolo, Max Ruder, Sebastian Locatelli, Acyr Kreutzer, Julia Frosst, Nick Gomez, Aidan Blunsom, Phil Fadaee, Marzieh Üstün, Ahmet Hooker, Sara Computation and Language This technical report introduces Aya 23, a family of multilingual language models. Aya 23 builds on the recent release of the Aya model (Üstün et al., 2024), focusing on pairing a highly performant pre-trained model with the recently released Aya collection (Singh et al., 2024). The result is a powerful multilingual large language model serving 23 languages, expanding state-of-art language modeling capabilities to approximately half of the world's population. The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs breadth, exploring the impact of allocating more capacity to fewer languages that are included during pre-training. Aya 23 outperforms both previous massively multilingual models like Aya 101 for the languages it covers, as well as widely used models like Gemma, Mistral and Mixtral on an extensive range of discriminative and generative tasks. We release the open weights for both the 8B and 35B models as part of our continued commitment for expanding access to multilingual progress. |
| title | Aya 23: Open Weight Releases to Further Multilingual Progress |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.15032 |