Where Are We? Evaluating LLM Performance on African Languages
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912412499705856 |
|---|---|
| author | Adebara, Ife Toyin, Hawau Olamide Ghebremichael, Nahom Tesfu Elmadany, AbdelRahim Abdul-Mageed, Muhammad |
| author_facet | Adebara, Ife Toyin, Hawau Olamide Ghebremichael, Nahom Tesfu Elmadany, AbdelRahim Abdul-Mageed, Muhammad |
| contents | Africa's rich linguistic heritage remains underrepresented in NLP, largely due to historical policies that favor foreign languages and create significant data inequities. In this paper, we integrate theoretical insights on Africa's language landscape with an empirical evaluation using Sahara - a comprehensive benchmark curated from large-scale, publicly accessible datasets capturing the continent's linguistic diversity. By systematically assessing the performance of leading large language models (LLMs) on Sahara, we demonstrate how policy-induced data variations directly impact model effectiveness across African languages. Our findings reveal that while a few languages perform reasonably well, many Indigenous languages remain marginalized due to sparse data. Leveraging these insights, we offer actionable recommendations for policy reforms and inclusive data practices. Overall, our work underscores the urgent need for a dual approach - combining theoretical understanding with empirical evaluation - to foster linguistic diversity in AI for African communities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_19582 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Where Are We? Evaluating LLM Performance on African Languages Adebara, Ife Toyin, Hawau Olamide Ghebremichael, Nahom Tesfu Elmadany, AbdelRahim Abdul-Mageed, Muhammad Computation and Language Africa's rich linguistic heritage remains underrepresented in NLP, largely due to historical policies that favor foreign languages and create significant data inequities. In this paper, we integrate theoretical insights on Africa's language landscape with an empirical evaluation using Sahara - a comprehensive benchmark curated from large-scale, publicly accessible datasets capturing the continent's linguistic diversity. By systematically assessing the performance of leading large language models (LLMs) on Sahara, we demonstrate how policy-induced data variations directly impact model effectiveness across African languages. Our findings reveal that while a few languages perform reasonably well, many Indigenous languages remain marginalized due to sparse data. Leveraging these insights, we offer actionable recommendations for policy reforms and inclusive data practices. Overall, our work underscores the urgent need for a dual approach - combining theoretical understanding with empirical evaluation - to foster linguistic diversity in AI for African communities. |
| title | Where Are We? Evaluating LLM Performance on African Languages |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.19582 |