Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916191386206208 |
|---|---|
| author | Bouthors, Maxime Crego, Josep Yvon, Francois |
| author_facet | Bouthors, Maxime Crego, Josep Yvon, Francois |
| contents | Retrieval-Augmented Neural Machine Translation (RAMT) architectures retrieve examples from memory to guide the generation process. While most works in this trend explore new ways to exploit the retrieved examples, the upstream retrieval step is mostly unexplored. In this paper, we study the effect of varying retrieval methods for several translation architectures, to better understand the interplay between these two processes. We conduct experiments in two language pairs in a multi-domain setting and consider several downstream architectures based on a standard autoregressive model, an edit-based model, and a large language model with in-context learning. Our experiments show that the choice of the retrieval technique impacts the translation scores, with variance across architectures. We also discuss the effects of increasing the number and diversity of examples, which are mostly positive across the board. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02835 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison Bouthors, Maxime Crego, Josep Yvon, Francois Computation and Language Retrieval-Augmented Neural Machine Translation (RAMT) architectures retrieve examples from memory to guide the generation process. While most works in this trend explore new ways to exploit the retrieved examples, the upstream retrieval step is mostly unexplored. In this paper, we study the effect of varying retrieval methods for several translation architectures, to better understand the interplay between these two processes. We conduct experiments in two language pairs in a multi-domain setting and consider several downstream architectures based on a standard autoregressive model, an edit-based model, and a large language model with in-context learning. Our experiments show that the choice of the retrieval technique impacts the translation scores, with variance across architectures. We also discuss the effects of increasing the number and diversity of examples, which are mostly positive across the board. |
| title | Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.02835 |