Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2606.01322 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918534356926464 |
|---|---|
| author | Akinode, Victor Li, Senyu Hamidouche, Wassim Zamir, Waqas Becker-Reshef, Inbal Adelani, David Ifeoluwa |
| author_facet | Akinode, Victor Li, Senyu Hamidouche, Wassim Zamir, Waqas Becker-Reshef, Inbal Adelani, David Ifeoluwa |
| contents | Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African languages that extends JailbreakBench (JBB) beyond direct translation through four settings: human translation of JBB prompts, English adaptation to African contexts followed by human translation, human-curated prompts validated through interactions with GPT-5.2, and code-switched prompts combining English and African languages, isolating the effect of language, cultural grounding, and prompt evasiveness on model safety. Across closed and open models, prompting in African languages reduces refusal relative to English, with culturally adapted prompts leading to least refusal. The evaluation also surfaces two structural limitations: model comprehension failures and reduced LLM-as-a-judge reliability in LRLs. To capture the first, we introduce Deflection alongside Refused and Jailbroken; to assess the second, we validate outputs with human annotations, showing that judge-human agreement drops in lower-resource languages and less commonly supported scripts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01322 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages Akinode, Victor Li, Senyu Hamidouche, Wassim Zamir, Waqas Becker-Reshef, Inbal Adelani, David Ifeoluwa Computation and Language Artificial Intelligence Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African languages that extends JailbreakBench (JBB) beyond direct translation through four settings: human translation of JBB prompts, English adaptation to African contexts followed by human translation, human-curated prompts validated through interactions with GPT-5.2, and code-switched prompts combining English and African languages, isolating the effect of language, cultural grounding, and prompt evasiveness on model safety. Across closed and open models, prompting in African languages reduces refusal relative to English, with culturally adapted prompts leading to least refusal. The evaluation also surfaces two structural limitations: model comprehension failures and reduced LLM-as-a-judge reliability in LRLs. To capture the first, we introduce Deflection alongside Refused and Jailbroken; to assess the second, we validate outputs with human annotations, showing that judge-human agreement drops in lower-resource languages and less commonly supported scripts. |
| title | TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2606.01322 |