Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909121763082240 |
|---|---|
| author | Alves, Duarte M. Pombal, José Guerreiro, Nuno M. Martins, Pedro H. Alves, João Farajian, Amin Peters, Ben Rei, Ricardo Fernandes, Patrick Agrawal, Sweta Colombo, Pierre de Souza, José G. C. Martins, André F. T. |
| author_facet | Alves, Duarte M. Pombal, José Guerreiro, Nuno M. Martins, Pedro H. Alves, João Farajian, Amin Peters, Ben Rei, Ricardo Fernandes, Patrick Agrawal, Sweta Colombo, Pierre de Souza, José G. C. Martins, André F. T. |
| contents | While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task. In this paper, we propose a recipe for tailoring LLMs to multiple tasks present in translation workflows. We perform continued pretraining on a multilingual mixture of monolingual and parallel data, creating TowerBase, followed by finetuning on instructions relevant for translation processes, creating TowerInstruct. Our final model surpasses open alternatives on several tasks relevant to translation workflows and is competitive with general-purpose closed LLMs. To facilitate future research, we release the Tower models, our specialization dataset, an evaluation framework for LLMs focusing on the translation ecosystem, and a collection of model generations, including ours, on our benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17733 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Tower: An Open Multilingual Large Language Model for Translation-Related Tasks Alves, Duarte M. Pombal, José Guerreiro, Nuno M. Martins, Pedro H. Alves, João Farajian, Amin Peters, Ben Rei, Ricardo Fernandes, Patrick Agrawal, Sweta Colombo, Pierre de Souza, José G. C. Martins, André F. T. Computation and Language While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task. In this paper, we propose a recipe for tailoring LLMs to multiple tasks present in translation workflows. We perform continued pretraining on a multilingual mixture of monolingual and parallel data, creating TowerBase, followed by finetuning on instructions relevant for translation processes, creating TowerInstruct. Our final model surpasses open alternatives on several tasks relevant to translation workflows and is competitive with general-purpose closed LLMs. To facilitate future research, we release the Tower models, our specialization dataset, an evaluation framework for LLMs focusing on the translation ecosystem, and a collection of model generations, including ours, on our benchmark. |
| title | Tower: An Open Multilingual Large Language Model for Translation-Related Tasks |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.17733 |