Relay Decoding: Concatenating Large Language Models for Machine Translation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910653492494336 |
|---|---|
| author | Fu, Chengpeng Feng, Xiaocheng Huang, Yichong Huo, Wenshuai Li, Baohang Wang, Hui Qin, Bin Liu, Ting |
| author_facet | Fu, Chengpeng Feng, Xiaocheng Huang, Yichong Huo, Wenshuai Li, Baohang Wang, Hui Qin, Bin Liu, Ting |
| contents | Leveraging large language models for machine translation has demonstrated promising results. However, it does require the large language models to possess the capability of handling both the source and target languages in machine translation. When it is challenging to find large models that support the desired languages, resorting to continuous learning methods becomes a costly endeavor. To mitigate these expenses, we propose an innovative approach called RD (Relay Decoding), which entails concatenating two distinct large models that individually support the source and target languages. By incorporating a simple mapping layer to facilitate the connection between these two models and utilizing a limited amount of parallel data for training, we successfully achieve superior results in the machine translation task. Experimental results conducted on the Multi30k and WikiMatrix datasets validate the effectiveness of our proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_02933 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Relay Decoding: Concatenating Large Language Models for Machine Translation Fu, Chengpeng Feng, Xiaocheng Huang, Yichong Huo, Wenshuai Li, Baohang Wang, Hui Qin, Bin Liu, Ting Computation and Language Leveraging large language models for machine translation has demonstrated promising results. However, it does require the large language models to possess the capability of handling both the source and target languages in machine translation. When it is challenging to find large models that support the desired languages, resorting to continuous learning methods becomes a costly endeavor. To mitigate these expenses, we propose an innovative approach called RD (Relay Decoding), which entails concatenating two distinct large models that individually support the source and target languages. By incorporating a simple mapping layer to facilitate the connection between these two models and utilizing a limited amount of parallel data for training, we successfully achieve superior results in the machine translation task. Experimental results conducted on the Multi30k and WikiMatrix datasets validate the effectiveness of our proposed method. |
| title | Relay Decoding: Concatenating Large Language Models for Machine Translation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.02933 |