MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916156957261824 |
|---|---|
| author | Mickus, Timothee Grönroos, Stig-Arne Attieh, Joseph Boggia, Michele De Gibert, Ona Ji, Shaoxiong Lopi, Niki Andreas Raganato, Alessandro Vázquez, Raúl Tiedemann, Jörg |
| author_facet | Mickus, Timothee Grönroos, Stig-Arne Attieh, Joseph Boggia, Michele De Gibert, Ona Ji, Shaoxiong Lopi, Niki Andreas Raganato, Alessandro Vázquez, Raúl Tiedemann, Jörg |
| contents | NLP in the age of monolithic large language models is approaching its limits in terms of size and information that can be handled. The trend goes to modularization, a necessary step into the direction of designing smaller sub-networks and components with specialized functionality. In this paper, we present the MAMMOTH toolkit: a framework designed for training massively multilingual modular machine translation systems at scale, initially derived from OpenNMT-py and then adapted to ensure efficient training across computation clusters. We showcase its efficiency across clusters of A100 and V100 NVIDIA GPUs, and discuss our design philosophy and plans for future information. The toolkit is publicly available online. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07544 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki Mickus, Timothee Grönroos, Stig-Arne Attieh, Joseph Boggia, Michele De Gibert, Ona Ji, Shaoxiong Lopi, Niki Andreas Raganato, Alessandro Vázquez, Raúl Tiedemann, Jörg Computation and Language NLP in the age of monolithic large language models is approaching its limits in terms of size and information that can be handled. The trend goes to modularization, a necessary step into the direction of designing smaller sub-networks and components with specialized functionality. In this paper, we present the MAMMOTH toolkit: a framework designed for training massively multilingual modular machine translation systems at scale, initially derived from OpenNMT-py and then adapted to ensure efficient training across computation clusters. We showcase its efficiency across clusters of A100 and V100 NVIDIA GPUs, and discuss our design philosophy and plans for future information. The toolkit is publicly available online. |
| title | MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.07544 |