Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912547757621248 |
|---|---|
| author | Banerjee, Somnath Layek, Sayan Chatterjee, Pratyush Mukherjee, Animesh Hazra, Rima |
| author_facet | Banerjee, Somnath Layek, Sayan Chatterjee, Pratyush Mukherjee, Animesh Hazra, Rima |
| contents | Ensuring consistent safety across multiple languages remains a significant challenge for large language models (LLMs). We introduce Soteria, a lightweight yet powerful strategy that locates and minimally adjusts the "functional heads" most responsible for harmful content generation in each language. By altering only a fraction of parameters, Soteria drastically reduces policy violations without sacrificing overall model performance, even in low-resource settings. To rigorously evaluate our approach, we also present XThreatBench, a specialized multilingual dataset capturing fine-grained harmful behaviors drawn from real policy guidelines. Experiments with leading open-source LLMs (e.g., Llama, Qwen, Mistral) show that Soteria consistently improves safety metrics across high-, mid-, and low-resource languages. These findings highlight a promising path toward scalable, linguistically attuned, and ethically aligned LLMs worldwide. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11244 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment Banerjee, Somnath Layek, Sayan Chatterjee, Pratyush Mukherjee, Animesh Hazra, Rima Computation and Language Artificial Intelligence Ensuring consistent safety across multiple languages remains a significant challenge for large language models (LLMs). We introduce Soteria, a lightweight yet powerful strategy that locates and minimally adjusts the "functional heads" most responsible for harmful content generation in each language. By altering only a fraction of parameters, Soteria drastically reduces policy violations without sacrificing overall model performance, even in low-resource settings. To rigorously evaluate our approach, we also present XThreatBench, a specialized multilingual dataset capturing fine-grained harmful behaviors drawn from real policy guidelines. Experiments with leading open-source LLMs (e.g., Llama, Qwen, Mistral) show that Soteria consistently improves safety metrics across high-, mid-, and low-resource languages. These findings highlight a promising path toward scalable, linguistically attuned, and ethically aligned LLMs worldwide. |
| title | Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.11244 |