Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908448529055744 |
|---|---|
| author | Goloburda, Maiya Laiyk, Nurkhan Turmakhan, Diana Wang, Yuxia Togmanov, Mukhammed Mansurov, Jonibek Sametov, Askhat Mukhituly, Nurdaulet Wang, Minghan Orel, Daniil Mujahid, Zain Muhammad Koto, Fajri Baldwin, Timothy Nakov, Preslav |
| author_facet | Goloburda, Maiya Laiyk, Nurkhan Turmakhan, Diana Wang, Yuxia Togmanov, Mukhammed Mansurov, Jonibek Sametov, Askhat Mukhituly, Nurdaulet Wang, Minghan Orel, Daniil Mujahid, Zain Muhammad Koto, Fajri Baldwin, Timothy Nakov, Preslav |
| contents | Large language models (LLMs) are known to have the potential to generate harmful content, posing risks to users. While significant progress has been made in developing taxonomies for LLM risks and safety evaluation prompts, most studies have focused on monolingual contexts, primarily in English. However, language- and region-specific risks in bilingual contexts are often overlooked, and core findings can diverge from those in monolingual settings. In this paper, we introduce Qorgau, a novel dataset specifically designed for safety evaluation in Kazakh and Russian, reflecting the unique bilingual context in Kazakhstan, where both Kazakh (a low-resource language) and Russian (a high-resource language) are spoken. Experiments with both multilingual and language-specific LLMs reveal notable differences in safety performance, emphasizing the need for tailored, region-specific datasets to ensure the responsible and safe deployment of LLMs in countries like Kazakhstan. Warning: this paper contains example data that may be offensive, harmful, or biased. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_13640 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts Goloburda, Maiya Laiyk, Nurkhan Turmakhan, Diana Wang, Yuxia Togmanov, Mukhammed Mansurov, Jonibek Sametov, Askhat Mukhituly, Nurdaulet Wang, Minghan Orel, Daniil Mujahid, Zain Muhammad Koto, Fajri Baldwin, Timothy Nakov, Preslav Computation and Language Large language models (LLMs) are known to have the potential to generate harmful content, posing risks to users. While significant progress has been made in developing taxonomies for LLM risks and safety evaluation prompts, most studies have focused on monolingual contexts, primarily in English. However, language- and region-specific risks in bilingual contexts are often overlooked, and core findings can diverge from those in monolingual settings. In this paper, we introduce Qorgau, a novel dataset specifically designed for safety evaluation in Kazakh and Russian, reflecting the unique bilingual context in Kazakhstan, where both Kazakh (a low-resource language) and Russian (a high-resource language) are spoken. Experiments with both multilingual and language-specific LLMs reveal notable differences in safety performance, emphasizing the need for tailored, region-specific datasets to ensure the responsible and safe deployment of LLMs in countries like Kazakhstan. Warning: this paper contains example data that may be offensive, harmful, or biased. |
| title | Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.13640 |