Learning from others' mistakes: Finetuning machine translation models with span-level error annotations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910660141514752 |
|---|---|
| author | Zhang, Lily H. Dadkhahi, Hamid Finkelstein, Mara Trabelsi, Firas Luo, Jiaming Freitag, Markus |
| author_facet | Zhang, Lily H. Dadkhahi, Hamid Finkelstein, Mara Trabelsi, Firas Luo, Jiaming Freitag, Markus |
| contents | Despite growing interest in incorporating feedback to improve language models, most efforts focus only on sequence-level annotations. In this work, we explore the potential of utilizing fine-grained span-level annotations from offline datasets to improve model quality. We develop a simple finetuning algorithm, called Training with Annotations (TWA), to directly train machine translation models on such annotated data. TWA utilizes targeted span-level error information while also flexibly learning what to penalize within a span. Moreover, TWA considers the overall trajectory of a sequence when deciding which non-error spans to utilize as positive signals. Experiments on English-German and Chinese-English machine translation show that TWA outperforms baselines such as Supervised FineTuning on sequences filtered for quality and Direct Preference Optimization on pairs constructed from the same data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16509 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning from others' mistakes: Finetuning machine translation models with span-level error annotations Zhang, Lily H. Dadkhahi, Hamid Finkelstein, Mara Trabelsi, Firas Luo, Jiaming Freitag, Markus Computation and Language Machine Learning Despite growing interest in incorporating feedback to improve language models, most efforts focus only on sequence-level annotations. In this work, we explore the potential of utilizing fine-grained span-level annotations from offline datasets to improve model quality. We develop a simple finetuning algorithm, called Training with Annotations (TWA), to directly train machine translation models on such annotated data. TWA utilizes targeted span-level error information while also flexibly learning what to penalize within a span. Moreover, TWA considers the overall trajectory of a sequence when deciding which non-error spans to utilize as positive signals. Experiments on English-German and Chinese-English machine translation show that TWA outperforms baselines such as Supervised FineTuning on sequences filtered for quality and Direct Preference Optimization on pairs constructed from the same data. |
| title | Learning from others' mistakes: Finetuning machine translation models with span-level error annotations |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2410.16509 |