Smoothie-Qwen: Post-Hoc Smoothing to Reduce Language Bias in Multilingual LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915376921575424 |
|---|---|
| author | Ji, SeungWon Lee, Jungyup Kim, Jemin Park, Sang Lee, SeungJae |
| author_facet | Ji, SeungWon Lee, Jungyup Kim, Jemin Park, Sang Lee, SeungJae |
| contents | Multilingual large language models (LLMs) often exhibit language confusion, a tendency to generate responses in a dominant language irrespective of the prompt's language. To address this, we propose Smoothie-Qwen, a lightweight, post-hoc method that mitigates language bias without retraining. This technique selectively adjusts token-level output probabilities to effectively suppress undesired language generation. Applied to the Qwen model, our method reduces unintended Chinese output by over 95% while preserving task accuracy on multilingual benchmarks. This work provides a practical and efficient solution for enhancing the language controllability of LLMs, making them more reliable for global applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05686 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Smoothie-Qwen: Post-Hoc Smoothing to Reduce Language Bias in Multilingual LLMs Ji, SeungWon Lee, Jungyup Kim, Jemin Park, Sang Lee, SeungJae Computation and Language Multilingual large language models (LLMs) often exhibit language confusion, a tendency to generate responses in a dominant language irrespective of the prompt's language. To address this, we propose Smoothie-Qwen, a lightweight, post-hoc method that mitigates language bias without retraining. This technique selectively adjusts token-level output probabilities to effectively suppress undesired language generation. Applied to the Qwen model, our method reduces unintended Chinese output by over 95% while preserving task accuracy on multilingual benchmarks. This work provides a practical and efficient solution for enhancing the language controllability of LLMs, making them more reliable for global applications. |
| title | Smoothie-Qwen: Post-Hoc Smoothing to Reduce Language Bias in Multilingual LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.05686 |