Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912067520299008 |
|---|---|
| author | Agrawal, Sweta de Souza, José G. C. Rei, Ricardo Farinhas, António Faria, Gonçalo Fernandes, Patrick Guerreiro, Nuno M Martins, Andre |
| author_facet | Agrawal, Sweta de Souza, José G. C. Rei, Ricardo Farinhas, António Faria, Gonçalo Fernandes, Patrick Guerreiro, Nuno M Martins, Andre |
| contents | Alignment with human preferences is an important step in developing accurate and safe large language models. This is no exception in machine translation (MT), where better handling of language nuances and context-specific variations leads to improved quality. However, preference data based on human feedback can be very expensive to obtain and curate at a large scale. Automatic metrics, on the other hand, can induce preferences, but they might not match human expectations perfectly. In this paper, we propose an approach that leverages the best of both worlds. We first collect sentence-level quality assessments from professional linguists on translations generated by multiple high-quality MT systems and evaluate the ability of current automatic metrics to recover these preferences. We then use this analysis to curate a new dataset, MT-Pref (metric induced translation preference) dataset, which comprises 18k instances covering 18 language directions, using texts sourced from multiple domains post-2022. We show that aligning TOWER models on MT-Pref significantly improves translation quality on WMT23 and FLORES benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_07779 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation Agrawal, Sweta de Souza, José G. C. Rei, Ricardo Farinhas, António Faria, Gonçalo Fernandes, Patrick Guerreiro, Nuno M Martins, Andre Computation and Language Alignment with human preferences is an important step in developing accurate and safe large language models. This is no exception in machine translation (MT), where better handling of language nuances and context-specific variations leads to improved quality. However, preference data based on human feedback can be very expensive to obtain and curate at a large scale. Automatic metrics, on the other hand, can induce preferences, but they might not match human expectations perfectly. In this paper, we propose an approach that leverages the best of both worlds. We first collect sentence-level quality assessments from professional linguists on translations generated by multiple high-quality MT systems and evaluate the ability of current automatic metrics to recover these preferences. We then use this analysis to curate a new dataset, MT-Pref (metric induced translation preference) dataset, which comprises 18k instances covering 18 language directions, using texts sourced from multiple domains post-2022. We show that aligning TOWER models on MT-Pref significantly improves translation quality on WMT23 and FLORES benchmarks. |
| title | Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.07779 |