Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.06374 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909842837340160 |
|---|---|
| author | Alabi, Jesujoba O. Azime, Israel Abebe Zhang, Miaoran España-Bonet, Cristina Bawden, Rachel Zhu, Dawei Adelani, David Ifeoluwa Odoje, Clement Oyeleke Akinade, Idris Maab, Iffat David, Davis Muhammad, Shamsuddeen Hassan Putini, Neo Ademuyiwa, David O. Caines, Andrew Klakow, Dietrich |
| author_facet | Alabi, Jesujoba O. Azime, Israel Abebe Zhang, Miaoran España-Bonet, Cristina Bawden, Rachel Zhu, Dawei Adelani, David Ifeoluwa Odoje, Clement Oyeleke Akinade, Idris Maab, Iffat David, Davis Muhammad, Shamsuddeen Hassan Putini, Neo Ademuyiwa, David O. Caines, Andrew Klakow, Dietrich |
| contents | This paper introduces AFRIDOC-MT, a document-level multi-parallel translation dataset covering English and five African languages: Amharic, Hausa, Swahili, Yorùbá, and Zulu. The dataset comprises 334 health and 271 information technology news documents, all human-translated from English to these languages. We conduct document-level translation benchmark experiments by evaluating neural machine translation (NMT) models and large language models (LLMs) for translations between English and these languages, at both the sentence and pseudo-document levels. These outputs are realigned to form complete documents for evaluation. Our results indicate that NLLB-200 achieved the best average performance among the standard NMT models, while GPT-4o outperformed general-purpose LLMs. Fine-tuning selected models led to substantial performance gains, but models trained on sentences struggled to generalize effectively to longer documents. Furthermore, our analysis reveals that some LLMs exhibit issues such as under-generation, repetition of words or phrases, and off-target translations, especially for African languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06374 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AFRIDOC-MT: Document-level MT Corpus for African Languages Alabi, Jesujoba O. Azime, Israel Abebe Zhang, Miaoran España-Bonet, Cristina Bawden, Rachel Zhu, Dawei Adelani, David Ifeoluwa Odoje, Clement Oyeleke Akinade, Idris Maab, Iffat David, Davis Muhammad, Shamsuddeen Hassan Putini, Neo Ademuyiwa, David O. Caines, Andrew Klakow, Dietrich Computation and Language This paper introduces AFRIDOC-MT, a document-level multi-parallel translation dataset covering English and five African languages: Amharic, Hausa, Swahili, Yorùbá, and Zulu. The dataset comprises 334 health and 271 information technology news documents, all human-translated from English to these languages. We conduct document-level translation benchmark experiments by evaluating neural machine translation (NMT) models and large language models (LLMs) for translations between English and these languages, at both the sentence and pseudo-document levels. These outputs are realigned to form complete documents for evaluation. Our results indicate that NLLB-200 achieved the best average performance among the standard NMT models, while GPT-4o outperformed general-purpose LLMs. Fine-tuning selected models led to substantial performance gains, but models trained on sentences struggled to generalize effectively to longer documents. Furthermore, our analysis reveals that some LLMs exhibit issues such as under-generation, repetition of words or phrases, and off-target translations, especially for African languages. |
| title | AFRIDOC-MT: Document-level MT Corpus for African Languages |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2501.06374 |