Pluggable Neural Machine Translation Models via Memory-augmented Adapters
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929283444768768 |
|---|---|
| author | Xu, Yuzhuang Wang, Shuo Li, Peng Liu, Xuebo Wang, Xiaolong Liu, Weidong Liu, Yang |
| author_facet | Xu, Yuzhuang Wang, Shuo Li, Peng Liu, Xuebo Wang, Xiaolong Liu, Weidong Liu, Yang |
| contents | Although neural machine translation (NMT) models perform well in the general domain, it remains rather challenging to control their generation behavior to satisfy the requirement of different users. Given the expensive training cost and the data scarcity challenge of learning a new model from scratch for each user requirement, we propose a memory-augmented adapter to steer pretrained NMT models in a pluggable manner. Specifically, we construct a multi-granular memory based on the user-provided text samples and propose a new adapter architecture to combine the model representations and the retrieved results. We also propose a training strategy using memory dropout to reduce spurious dependencies between the NMT model and the memory. We validate our approach on both style- and domain-specific experiments and the results indicate that our method can outperform several representative pluggable baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_06029 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Pluggable Neural Machine Translation Models via Memory-augmented Adapters Xu, Yuzhuang Wang, Shuo Li, Peng Liu, Xuebo Wang, Xiaolong Liu, Weidong Liu, Yang Computation and Language Although neural machine translation (NMT) models perform well in the general domain, it remains rather challenging to control their generation behavior to satisfy the requirement of different users. Given the expensive training cost and the data scarcity challenge of learning a new model from scratch for each user requirement, we propose a memory-augmented adapter to steer pretrained NMT models in a pluggable manner. Specifically, we construct a multi-granular memory based on the user-provided text samples and propose a new adapter architecture to combine the model representations and the retrieved results. We also propose a training strategy using memory dropout to reduce spurious dependencies between the NMT model and the memory. We validate our approach on both style- and domain-specific experiments and the results indicate that our method can outperform several representative pluggable baselines. |
| title | Pluggable Neural Machine Translation Models via Memory-augmented Adapters |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2307.06029 |