Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912255730253824 |
|---|---|
| author | Zhao, Yiran Liu, Chaoqun Deng, Yue Ying, Jiahao Aljunied, Mahani Li, Zhaodonghui Bing, Lidong Chan, Hou Pong Rong, Yu Zhao, Deli Zhang, Wenxuan |
| author_facet | Zhao, Yiran Liu, Chaoqun Deng, Yue Ying, Jiahao Aljunied, Mahani Li, Zhaodonghui Bing, Lidong Chan, Hou Pong Rong, Yu Zhao, Deli Zhang, Wenxuan |
| contents | Large language models (LLMs) have revolutionized natural language processing (NLP), yet open-source multilingual LLMs remain scarce, with existing models often limited in language coverage. Such models typically prioritize well-resourced languages, while widely spoken but under-resourced languages are often overlooked. To address this disparity, we introduce $\texttt{Babel}$, an open multilingual LLM that covers the top 25 languages by number of speakers, supports over 90% of the global population, and includes many languages neglected by other open multilingual LLMs. Unlike traditional continue pretraining approaches, Babel expands its parameter count through a layer extension technique that elevates Babel's performance ceiling. We introduce two variants: $\texttt{Babel-9B}$, designed for efficient inference and fine-tuning, and $\texttt{Babel-83B}$, which sets a new standard for open multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its superior performance compared to open LLMs of comparable size. In addition, using open-source supervised fine-tuning datasets, Babel achieves remarkable performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat setting a new standard for multilingual tasks, reaching the same level of commercial models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00865 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakers Zhao, Yiran Liu, Chaoqun Deng, Yue Ying, Jiahao Aljunied, Mahani Li, Zhaodonghui Bing, Lidong Chan, Hou Pong Rong, Yu Zhao, Deli Zhang, Wenxuan Computation and Language Artificial Intelligence Large language models (LLMs) have revolutionized natural language processing (NLP), yet open-source multilingual LLMs remain scarce, with existing models often limited in language coverage. Such models typically prioritize well-resourced languages, while widely spoken but under-resourced languages are often overlooked. To address this disparity, we introduce $\texttt{Babel}$, an open multilingual LLM that covers the top 25 languages by number of speakers, supports over 90% of the global population, and includes many languages neglected by other open multilingual LLMs. Unlike traditional continue pretraining approaches, Babel expands its parameter count through a layer extension technique that elevates Babel's performance ceiling. We introduce two variants: $\texttt{Babel-9B}$, designed for efficient inference and fine-tuning, and $\texttt{Babel-83B}$, which sets a new standard for open multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its superior performance compared to open LLMs of comparable size. In addition, using open-source supervised fine-tuning datasets, Babel achieves remarkable performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat setting a new standard for multilingual tasks, reaching the same level of commercial models. |
| title | Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakers |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.00865 |