Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917638624509952 |
|---|---|
| author | Li, Jiahuan Zhou, Hao Huang, Shujian Cheng, Shanbo Chen, Jiajun |
| author_facet | Li, Jiahuan Zhou, Hao Huang, Shujian Cheng, Shanbo Chen, Jiajun |
| contents | Large-scale Pretrained Language Models (LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language, the performance depends on its similarity to English and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instructions and the alignment among different languages. With multilingual finetuning, LLMs could learn to perform the translation task well even for those language pairs unseen during the instruction tuning phase. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_15083 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions Li, Jiahuan Zhou, Hao Huang, Shujian Cheng, Shanbo Chen, Jiajun Computation and Language Large-scale Pretrained Language Models (LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language, the performance depends on its similarity to English and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instructions and the alignment among different languages. With multilingual finetuning, LLMs could learn to perform the translation task well even for those language pairs unseen during the instruction tuning phase. |
| title | Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2305.15083 |