A Framework to Assess Multilingual Vulnerabilities of LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929762171092992 |
|---|---|
| author | Tang, Likai Bogahawatta, Niruth Ginige, Yasod Xu, Jiarui Sun, Shixuan Ranathunga, Surangika Seneviratne, Suranga |
| author_facet | Tang, Likai Bogahawatta, Niruth Ginige, Yasod Xu, Jiarui Sun, Shixuan Ranathunga, Surangika Seneviratne, Suranga |
| contents | Large Language Models (LLMs) are acquiring a wider range of capabilities, including understanding and responding in multiple languages. While they undergo safety training to prevent them from answering illegal questions, imbalances in training data and human evaluation resources can make these models more susceptible to attacks in low-resource languages (LRL). This paper proposes a framework to automatically assess the multilingual vulnerabilities of commonly used LLMs. Using our framework, we evaluated six LLMs across eight languages representing varying levels of resource availability. We validated the assessments generated by our automated framework through human evaluation in two languages, demonstrating that the framework's results align with human judgments in most cases. Our findings reveal vulnerabilities in LRL; however, these may pose minimal risk as they often stem from the model's poor performance, resulting in incoherent responses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_13081 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Framework to Assess Multilingual Vulnerabilities of LLMs Tang, Likai Bogahawatta, Niruth Ginige, Yasod Xu, Jiarui Sun, Shixuan Ranathunga, Surangika Seneviratne, Suranga Computation and Language Artificial Intelligence Large Language Models (LLMs) are acquiring a wider range of capabilities, including understanding and responding in multiple languages. While they undergo safety training to prevent them from answering illegal questions, imbalances in training data and human evaluation resources can make these models more susceptible to attacks in low-resource languages (LRL). This paper proposes a framework to automatically assess the multilingual vulnerabilities of commonly used LLMs. Using our framework, we evaluated six LLMs across eight languages representing varying levels of resource availability. We validated the assessments generated by our automated framework through human evaluation in two languages, demonstrating that the framework's results align with human judgments in most cases. Our findings reveal vulnerabilities in LRL; however, these may pose minimal risk as they often stem from the model's poor performance, resulting in incoherent responses. |
| title | A Framework to Assess Multilingual Vulnerabilities of LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.13081 |