Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages -- A Singlish Case Study
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916677968461824 |
|---|---|
| author | Lim, Isaac Khoo, Shaun Lee, Roy Ka-Wei Chua, Watson Goh, Jia Yi Foo, Jessica |
| author_facet | Lim, Isaac Khoo, Shaun Lee, Roy Ka-Wei Chua, Watson Goh, Jia Yi Foo, Jessica |
| contents | Ensuring the safety of Large Language Models (LLMs) in diverse linguistic settings remains challenging, particularly for low-resource languages. Existing safety alignment methods are English-centric, limiting their effectiveness. We systematically compare Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO) for aligning SEA-Lion-v2.1-Instruct, a Llama 3-8B variant, to reduce toxicity in Singlish. Our results show that SFT+KTO achieves superior safety alignment with higher sample efficiency than DPO. Additionally, we introduce KTO-S, which enhances stability via improved KL divergence regularization. Our approach reduces Singlish toxicity by 99\%, generalizes to TOXIGEN, and maintains strong performance on standard LLM benchmarks, providing a scalable framework for safer AI deployment in multilingual contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12485 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages -- A Singlish Case Study Lim, Isaac Khoo, Shaun Lee, Roy Ka-Wei Chua, Watson Goh, Jia Yi Foo, Jessica Computation and Language Artificial Intelligence Ensuring the safety of Large Language Models (LLMs) in diverse linguistic settings remains challenging, particularly for low-resource languages. Existing safety alignment methods are English-centric, limiting their effectiveness. We systematically compare Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO) for aligning SEA-Lion-v2.1-Instruct, a Llama 3-8B variant, to reduce toxicity in Singlish. Our results show that SFT+KTO achieves superior safety alignment with higher sample efficiency than DPO. Additionally, we introduce KTO-S, which enhances stability via improved KL divergence regularization. Our approach reduces Singlish toxicity by 99\%, generalizes to TOXIGEN, and maintains strong performance on standard LLM benchmarks, providing a scalable framework for safer AI deployment in multilingual contexts. |
| title | Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages -- A Singlish Case Study |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.12485 |