Multilingual Large Language Models do not comprehend all natural languages to equal degrees
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911463143112704 |
|---|---|
| author | Moskvina, Natalia Montero, Raquel Yoshida, Masaya Hubers, Ferdy Morosi, Paolo Irhaymi, Walid Yan, Jin Serrano, Tamara Pagliarini, Elena Günther, Fritz Leivada, Evelina |
| author_facet | Moskvina, Natalia Montero, Raquel Yoshida, Masaya Hubers, Ferdy Morosi, Paolo Irhaymi, Walid Yan, Jin Serrano, Tamara Pagliarini, Elena Günther, Fritz Leivada, Evelina |
| contents | Large Language Models (LLMs) play a critical role in how humans access information. While their core use relies on comprehending written requests, our understanding of this ability is currently limited, because most benchmarks evaluate LLMs in high-resource languages predominantly spoken by Western, Educated, Industrialised, Rich, and Democratic (WEIRD) communities. The default assumption is that English is the best-performing language for LLMs, while smaller, low-resource languages are linked to less reliable outputs, even in multilingual, state-of-the-art models. To track variation in the comprehension abilities of LLMs, we prompt 3 popular models on a language comprehension task across 12 languages, representing the Indo-European, Afro-Asiatic, Turkic, Sino-Tibetan, and Japonic language families. Our results suggest that the models exhibit remarkable linguistic accuracy across typologically diverse languages, yet they fall behind human baselines in all of them, albeit to different degrees. Contrary to what was expected, English is not the best-performing language, as it was systematically outperformed by several Romance languages, even lower-resource ones. We frame the results by discussing the role of several factors that drive LLM performance, such as tokenization, language distance from Spanish and English, size of training data, and data origin in high- vs. low-resource languages and WEIRD vs. non-WEIRD communities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_20065 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Multilingual Large Language Models do not comprehend all natural languages to equal degrees Moskvina, Natalia Montero, Raquel Yoshida, Masaya Hubers, Ferdy Morosi, Paolo Irhaymi, Walid Yan, Jin Serrano, Tamara Pagliarini, Elena Günther, Fritz Leivada, Evelina Computation and Language Artificial Intelligence Large Language Models (LLMs) play a critical role in how humans access information. While their core use relies on comprehending written requests, our understanding of this ability is currently limited, because most benchmarks evaluate LLMs in high-resource languages predominantly spoken by Western, Educated, Industrialised, Rich, and Democratic (WEIRD) communities. The default assumption is that English is the best-performing language for LLMs, while smaller, low-resource languages are linked to less reliable outputs, even in multilingual, state-of-the-art models. To track variation in the comprehension abilities of LLMs, we prompt 3 popular models on a language comprehension task across 12 languages, representing the Indo-European, Afro-Asiatic, Turkic, Sino-Tibetan, and Japonic language families. Our results suggest that the models exhibit remarkable linguistic accuracy across typologically diverse languages, yet they fall behind human baselines in all of them, albeit to different degrees. Contrary to what was expected, English is not the best-performing language, as it was systematically outperformed by several Romance languages, even lower-resource ones. We frame the results by discussing the role of several factors that drive LLM performance, such as tokenization, language distance from Spanish and English, size of training data, and data origin in high- vs. low-resource languages and WEIRD vs. non-WEIRD communities. |
| title | Multilingual Large Language Models do not comprehend all natural languages to equal degrees |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.20065 |