Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914399039520768 |
|---|---|
| author | Toyin, Hawau Olamide Magdy, Samar Mohamed Aldarmaki, Hanan |
| author_facet | Toyin, Hawau Olamide Magdy, Samar Mohamed Aldarmaki, Hanan |
| contents | We investigate the effectiveness of large language models (LLMs) for text diacritization in two typologically distinct languages: Arabic and Yoruba. To enable a rigorous evaluation, we introduce a novel multilingual dataset MultiDiac, with diverse samples that capture a range of diacritic ambiguities. We evaluate 12 LLMs varying in size, accessibility, and language coverage, and benchmark them against $4$ specialized diacritization models. Additionally, we fine-tune four small open-source models using LoRA for Yoruba. Our results show that many off-the-shelf LLMs outperform specialized diacritization models, but smaller models suffer from hallucinations. We find that fine-tuning on a small dataset can help improve diacritization performance and reduce hallucinations for Yoruba. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_11602 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study Toyin, Hawau Olamide Magdy, Samar Mohamed Aldarmaki, Hanan Computation and Language Artificial Intelligence We investigate the effectiveness of large language models (LLMs) for text diacritization in two typologically distinct languages: Arabic and Yoruba. To enable a rigorous evaluation, we introduce a novel multilingual dataset MultiDiac, with diverse samples that capture a range of diacritic ambiguities. We evaluate 12 LLMs varying in size, accessibility, and language coverage, and benchmark them against $4$ specialized diacritization models. Additionally, we fine-tune four small open-source models using LoRA for Yoruba. Our results show that many off-the-shelf LLMs outperform specialized diacritization models, but smaller models suffer from hallucinations. We find that fine-tuning on a small dataset can help improve diacritization performance and reduce hallucinations for Yoruba. |
| title | Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.11602 |