Understanding and Mitigating the Uncertainty in Zero-Shot Translation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910656952795136 |
|---|---|
| author | Wang, Wenxuan Jiao, Wenxiang Wang, Shuo Tu, Zhaopeng Lyu, Michael R. |
| author_facet | Wang, Wenxuan Jiao, Wenxiang Wang, Shuo Tu, Zhaopeng Lyu, Michael R. |
| contents | Zero-shot translation is a promising direction for building a comprehensive multilingual neural machine translation~(MNMT) system. However, its quality is still not satisfactory due to off-target issues. In this paper, we aim to understand and alleviate the off-target issues from the perspective of uncertainty in zero-shot translation. By carefully examining the translation output and model confidence, we identify two uncertainties that are responsible for the off-target issues, namely, extrinsic data uncertainty and intrinsic model uncertainty. Based on the observations, we propose two lightweight and complementary approaches to denoise the training data for model training and explicitly penalize the off-target translations by unlikelihood training during model training. Extensive experiments on both balanced and imbalanced datasets show that our approaches significantly improve the performance of zero-shot translation over strong MNMT baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2205_10068 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Understanding and Mitigating the Uncertainty in Zero-Shot Translation Wang, Wenxuan Jiao, Wenxiang Wang, Shuo Tu, Zhaopeng Lyu, Michael R. Computation and Language Artificial Intelligence Machine Learning Zero-shot translation is a promising direction for building a comprehensive multilingual neural machine translation~(MNMT) system. However, its quality is still not satisfactory due to off-target issues. In this paper, we aim to understand and alleviate the off-target issues from the perspective of uncertainty in zero-shot translation. By carefully examining the translation output and model confidence, we identify two uncertainties that are responsible for the off-target issues, namely, extrinsic data uncertainty and intrinsic model uncertainty. Based on the observations, we propose two lightweight and complementary approaches to denoise the training data for model training and explicitly penalize the off-target translations by unlikelihood training during model training. Extensive experiments on both balanced and imbalanced datasets show that our approaches significantly improve the performance of zero-shot translation over strong MNMT baselines. |
| title | Understanding and Mitigating the Uncertainty in Zero-Shot Translation |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2205.10068 |