Investigating the translation capabilities of Large Language Models trained on parallel data only
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911916050350080 |
|---|---|
| author | Gilabert, Javier García Escolano, Carlos Savall, Aleix Sant Fornaciari, Francesca De Luca Mash, Audrey Liao, Xixian Melero, Maite |
| author_facet | Gilabert, Javier García Escolano, Carlos Savall, Aleix Sant Fornaciari, Francesca De Luca Mash, Audrey Liao, Xixian Melero, Maite |
| contents | In recent years, Large Language Models (LLMs) have demonstrated exceptional proficiency across a broad spectrum of Natural Language Processing (NLP) tasks, including Machine Translation. However, previous methods predominantly relied on iterative processes such as instruction fine-tuning or continual pre-training, leaving unexplored the challenges of training LLMs solely on parallel data. In this work, we introduce PLUME (Parallel Language Model), a collection of three 2B LLMs featuring varying vocabulary sizes (32k, 128k, and 256k) trained exclusively on Catalan-centric parallel examples. These models perform comparably to previous encoder-decoder architectures on 16 supervised translation directions and 56 zero-shot ones. Utilizing this set of models, we conduct a thorough investigation into the translation capabilities of LLMs, probing their performance, the impact of the different elements of the prompt, and their cross-lingual representation space. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09140 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Investigating the translation capabilities of Large Language Models trained on parallel data only Gilabert, Javier García Escolano, Carlos Savall, Aleix Sant Fornaciari, Francesca De Luca Mash, Audrey Liao, Xixian Melero, Maite Computation and Language In recent years, Large Language Models (LLMs) have demonstrated exceptional proficiency across a broad spectrum of Natural Language Processing (NLP) tasks, including Machine Translation. However, previous methods predominantly relied on iterative processes such as instruction fine-tuning or continual pre-training, leaving unexplored the challenges of training LLMs solely on parallel data. In this work, we introduce PLUME (Parallel Language Model), a collection of three 2B LLMs featuring varying vocabulary sizes (32k, 128k, and 256k) trained exclusively on Catalan-centric parallel examples. These models perform comparably to previous encoder-decoder architectures on 16 supervised translation directions and 56 zero-shot ones. Utilizing this set of models, we conduct a thorough investigation into the translation capabilities of LLMs, probing their performance, the impact of the different elements of the prompt, and their cross-lingual representation space. |
| title | Investigating the translation capabilities of Large Language Models trained on parallel data only |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.09140 |