The State of Large Language Models for African Languages: Progress and Challenges
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911022444445696 |
|---|---|
| author | Hussen, Kedir Yassin Sewunetie, Walelign Tewabe Ayele, Abinew Ali Imam, Sukairaj Hafiz Muhammad, Shamsuddeen Hassan Yimam, Seid Muhie |
| author_facet | Hussen, Kedir Yassin Sewunetie, Walelign Tewabe Ayele, Abinew Ali Imam, Sukairaj Hafiz Muhammad, Shamsuddeen Hassan Yimam, Seid Muhie |
| contents | Large Language Models (LLMs) are transforming Natural Language Processing (NLP), but their benefits are largely absent for Africa's 2,000 low-resource languages. This paper comparatively analyzes African language coverage across six LLMs, eight Small Language Models (SLMs), and six Specialized SLMs (SSLMs). The evaluation covers language coverage, training sets, technical limitations, script problems, and language modelling roadmaps. The work identifies 42 supported African languages and 23 available public data sets, and it shows a big gap where four languages (Amharic, Swahili, Afrikaans, and Malagasy) are always treated while there is over 98\% of unsupported African languages. Moreover, the review shows that just Latin, Arabic, and Ge'ez scripts are identified while 20 active scripts are neglected. Some of the primary challenges are lack of data, tokenization biases, computational costs being very high, and evaluation issues. These issues demand language standardization, corpus development by the community, and effective adaptation methods for African languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_02280 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The State of Large Language Models for African Languages: Progress and Challenges Hussen, Kedir Yassin Sewunetie, Walelign Tewabe Ayele, Abinew Ali Imam, Sukairaj Hafiz Muhammad, Shamsuddeen Hassan Yimam, Seid Muhie Artificial Intelligence Large Language Models (LLMs) are transforming Natural Language Processing (NLP), but their benefits are largely absent for Africa's 2,000 low-resource languages. This paper comparatively analyzes African language coverage across six LLMs, eight Small Language Models (SLMs), and six Specialized SLMs (SSLMs). The evaluation covers language coverage, training sets, technical limitations, script problems, and language modelling roadmaps. The work identifies 42 supported African languages and 23 available public data sets, and it shows a big gap where four languages (Amharic, Swahili, Afrikaans, and Malagasy) are always treated while there is over 98\% of unsupported African languages. Moreover, the review shows that just Latin, Arabic, and Ge'ez scripts are identified while 20 active scripts are neglected. Some of the primary challenges are lack of data, tokenization biases, computational costs being very high, and evaluation issues. These issues demand language standardization, corpus development by the community, and effective adaptation methods for African languages. |
| title | The State of Large Language Models for African Languages: Progress and Challenges |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.02280 |