Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911423952584704 |
|---|---|
| author | Abouzahir, Chaimae Ma, Congbo Habash, Nizar Shamout, Farah E. |
| author_facet | Abouzahir, Chaimae Ma, Congbo Habash, Nizar Shamout, Farah E. |
| contents | In recent years, Large Language Models (LLMs) have become widely used in medical applications, such as clinical decision support, medical education, and medical question answering. Yet, these models are often English-centric, limiting their robustness and reliability for linguistically diverse communities. Recent work has highlighted discrepancies in performance in low-resource languages for various medical tasks, but the underlying causes remain poorly understood. In this study, we conduct a cross-lingual empirical analysis of LLM performance on Arabic and English medical question and answering. Our findings reveal a persistent language-driven performance gap that intensifies with increasing task complexity. Tokenization analysis exposes structural fragmentation in Arabic medical text, while reliability analysis suggests that model-reported confidence and explanations exhibit limited correlation with correctness. Together, these findings underscore the need for language-aware design and evaluation strategies in LLMs for medical tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05374 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks Abouzahir, Chaimae Ma, Congbo Habash, Nizar Shamout, Farah E. Computation and Language Machine Learning In recent years, Large Language Models (LLMs) have become widely used in medical applications, such as clinical decision support, medical education, and medical question answering. Yet, these models are often English-centric, limiting their robustness and reliability for linguistically diverse communities. Recent work has highlighted discrepancies in performance in low-resource languages for various medical tasks, but the underlying causes remain poorly understood. In this study, we conduct a cross-lingual empirical analysis of LLM performance on Arabic and English medical question and answering. Our findings reveal a persistent language-driven performance gap that intensifies with increasing task complexity. Tokenization analysis exposes structural fragmentation in Arabic medical text, while reliability analysis suggests that model-reported confidence and explanations exhibit limited correlation with correctness. Together, these findings underscore the need for language-aware design and evaluation strategies in LLMs for medical tasks. |
| title | Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.05374 |