Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.13286 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910130116755456 |
|---|---|
| author | Dhaliwal, Mehak Chaurasia, Shashwat Qin, Yao Hong, Dezhi Butler, Thomas |
| author_facet | Dhaliwal, Mehak Chaurasia, Shashwat Qin, Yao Hong, Dezhi Butler, Thomas |
| contents | Despite the widespread multilingual deployment of large language models, post-training pipelines remain predominantly English-centric, contributing to performance disparities across languages. We present a systematic, controlled study of the interplay between training language coverage, model scale, and task domain, based on 220 supervised fine-tuning runs on parallel translated multilingual data mixtures spanning mathematical reasoning and API calling tasks, with models up to 8B parameters. We find that increasing language coverage during post-training is largely beneficial across tasks and model scales, with low-resource languages benefiting the most and high-resource languages plateauing rather than degrading. Even minimal multilinguality helps: incorporating a single non-English language improves both English performance and cross-lingual generalization, making English-only post-training largely suboptimal. Moreover, at sufficient language diversity, zero-shot cross-lingual transfer can match or exceed the effects of direct language inclusion in a low-diversity setting, although gains remain limited for typologically distant, low-resource languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_13286 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training Dhaliwal, Mehak Chaurasia, Shashwat Qin, Yao Hong, Dezhi Butler, Thomas Computation and Language Artificial Intelligence Despite the widespread multilingual deployment of large language models, post-training pipelines remain predominantly English-centric, contributing to performance disparities across languages. We present a systematic, controlled study of the interplay between training language coverage, model scale, and task domain, based on 220 supervised fine-tuning runs on parallel translated multilingual data mixtures spanning mathematical reasoning and API calling tasks, with models up to 8B parameters. We find that increasing language coverage during post-training is largely beneficial across tasks and model scales, with low-resource languages benefiting the most and high-resource languages plateauing rather than degrading. Even minimal multilinguality helps: incorporating a single non-English language improves both English performance and cross-lingual generalization, making English-only post-training largely suboptimal. Moreover, at sufficient language diversity, zero-shot cross-lingual transfer can match or exceed the effects of direct language inclusion in a low-diversity setting, although gains remain limited for typologically distant, low-resource languages. |
| title | English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2604.13286 |