Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915259605843968 |
|---|---|
| author | Buscemi, Alessio Lothritz, Cédric Morales, Sergio Gomez-Vazquez, Marcos Clarisó, Robert Cabot, Jordi Castignani, German |
| author_facet | Buscemi, Alessio Lothritz, Cédric Morales, Sergio Gomez-Vazquez, Marcos Clarisó, Robert Cabot, Jordi Castignani, German |
| contents | Large Language Models (LLMs) have exhibited impressive natural language processing capabilities but often perpetuate social biases inherent in their training data. To address this, we introduce MultiLingual Augmented Bias Testing (MLA-BiTe), a framework that improves prior bias evaluation methods by enabling systematic multilingual bias testing. MLA-BiTe leverages automated translation and paraphrasing techniques to support comprehensive assessments across diverse linguistic settings. In this study, we evaluate the effectiveness of MLA-BiTe by testing four state-of-the-art LLMs in six languages -- including two low-resource languages -- focusing on seven sensitive categories of discrimination. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18560 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages Buscemi, Alessio Lothritz, Cédric Morales, Sergio Gomez-Vazquez, Marcos Clarisó, Robert Cabot, Jordi Castignani, German Computation and Language Artificial Intelligence Large Language Models (LLMs) have exhibited impressive natural language processing capabilities but often perpetuate social biases inherent in their training data. To address this, we introduce MultiLingual Augmented Bias Testing (MLA-BiTe), a framework that improves prior bias evaluation methods by enabling systematic multilingual bias testing. MLA-BiTe leverages automated translation and paraphrasing techniques to support comprehensive assessments across diverse linguistic settings. In this study, we evaluate the effectiveness of MLA-BiTe by testing four state-of-the-art LLMs in six languages -- including two low-resource languages -- focusing on seven sensitive categories of discrimination. |
| title | Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.18560 |