Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910407639171072 |
|---|---|
| author | Yang, Guangyu Chen, Jinghong Lin, Weizhe Byrne, Bill |
| author_facet | Yang, Guangyu Chen, Jinghong Lin, Weizhe Byrne, Bill |
| contents | Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_08380 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding Yang, Guangyu Chen, Jinghong Lin, Weizhe Byrne, Bill Computation and Language Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO. |
| title | Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2311.08380 |