Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917574723239936 |
|---|---|
| author | Vamvas, Jannis Aepli, Noëmi Sennrich, Rico |
| author_facet | Vamvas, Jannis Aepli, Noëmi Sennrich, Rico |
| contents | Creating neural text encoders for written Swiss German is challenging due to a dearth of training data combined with dialectal variation. In this paper, we build on several existing multilingual encoders and adapt them to Swiss German using continued pre-training. Evaluation on three diverse downstream tasks shows that simply adding a Swiss German adapter to a modular encoder achieves 97.5% of fully monolithic adaptation performance. We further find that for the task of retrieving Swiss German sentences given Standard German queries, adapting a character-level model is more effective than the other adaptation strategies. We release our code and the models trained for our experiments at https://github.com/ZurichNLP/swiss-german-text-encoders |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14400 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect Vamvas, Jannis Aepli, Noëmi Sennrich, Rico Computation and Language Creating neural text encoders for written Swiss German is challenging due to a dearth of training data combined with dialectal variation. In this paper, we build on several existing multilingual encoders and adapt them to Swiss German using continued pre-training. Evaluation on three diverse downstream tasks shows that simply adding a Swiss German adapter to a modular encoder achieves 97.5% of fully monolithic adaptation performance. We further find that for the task of retrieving Swiss German sentences given Standard German queries, adapting a character-level model is more effective than the other adaptation strategies. We release our code and the models trained for our experiments at https://github.com/ZurichNLP/swiss-german-text-encoders |
| title | Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.14400 |