Cover Image

Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dou, Longxu, Liu, Qian, Zhou, Fan, Chen, Changyu, Wang, Zili, Jin, Ziqi, Liu, Zichen, Zhu, Tongyao, Du, Cunxiao, Yang, Penghui, Wang, Haonan, Liu, Jiaheng, Zhao, Yongchi, Feng, Xiachong, Mao, Xin, Yeung, Man Tsung, Pipatanakul, Kunat, Koto, Fajri, Thu, Min Si, Kydlíček, Hynek, Liu, Zeyi, Lin, Qunshu, Sripaisarnmongkol, Sittipong, Sae-Khow, Kridtaphad, Thongchim, Nirattisai, Konkaew, Taechawat, Borijindargoon, Narong, Dao, Anh, Maneegard, Matichon, Artkaew, Phakphum, Yong, Zheng-Xin, Nguyen, Quan, Phatthiyaphaibun, Wannaphong, Tran, Hoang H., Zhang, Mike, Chen, Shiqi, Pang, Tianyu, Du, Chao, Wan, Xinyi, Lu, Wei, Lin, Min
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866915158407774208
author	Dou, Longxu Liu, Qian Zhou, Fan Chen, Changyu Wang, Zili Jin, Ziqi Liu, Zichen Zhu, Tongyao Du, Cunxiao Yang, Penghui Wang, Haonan Liu, Jiaheng Zhao, Yongchi Feng, Xiachong Mao, Xin Yeung, Man Tsung Pipatanakul, Kunat Koto, Fajri Thu, Min Si Kydlíček, Hynek Liu, Zeyi Lin, Qunshu Sripaisarnmongkol, Sittipong Sae-Khow, Kridtaphad Thongchim, Nirattisai Konkaew, Taechawat Borijindargoon, Narong Dao, Anh Maneegard, Matichon Artkaew, Phakphum Yong, Zheng-Xin Nguyen, Quan Phatthiyaphaibun, Wannaphong Tran, Hoang H. Zhang, Mike Chen, Shiqi Pang, Tianyu Du, Chao Wan, Xinyi Lu, Wei Lin, Min
author_facet	Dou, Longxu Liu, Qian Zhou, Fan Chen, Changyu Wang, Zili Jin, Ziqi Liu, Zichen Zhu, Tongyao Du, Cunxiao Yang, Penghui Wang, Haonan Liu, Jiaheng Zhao, Yongchi Feng, Xiachong Mao, Xin Yeung, Man Tsung Pipatanakul, Kunat Koto, Fajri Thu, Min Si Kydlíček, Hynek Liu, Zeyi Lin, Qunshu Sripaisarnmongkol, Sittipong Sae-Khow, Kridtaphad Thongchim, Nirattisai Konkaew, Taechawat Borijindargoon, Narong Dao, Anh Maneegard, Matichon Artkaew, Phakphum Yong, Zheng-Xin Nguyen, Quan Phatthiyaphaibun, Wannaphong Tran, Hoang H. Zhang, Mike Chen, Shiqi Pang, Tianyu Du, Chao Wan, Xinyi Lu, Wei Lin, Min
contents	Sailor2 is a family of cutting-edge multilingual language models for South-East Asian (SEA) languages, available in 1B, 8B, and 20B sizes to suit diverse applications. Building on Qwen2.5, Sailor2 undergoes continuous pre-training on 500B tokens (400B SEA-specific and 100B replay tokens) to support 13 SEA languages while retaining proficiency in Chinese and English. Sailor2-20B model achieves a 50-50 win rate against GPT-4o across SEA languages. We also deliver a comprehensive cookbook on how to develop the multilingual model in an efficient manner, including five key aspects: data curation, pre-training, post-training, model customization and evaluation. We hope that Sailor2 model (Apache 2.0 license) will drive language development in the SEA region, and Sailor2 cookbook will inspire researchers to build more inclusive LLMs for other under-served languages.
format	Preprint
id	arxiv_https___arxiv_org_abs_2502_12982
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs Dou, Longxu Liu, Qian Zhou, Fan Chen, Changyu Wang, Zili Jin, Ziqi Liu, Zichen Zhu, Tongyao Du, Cunxiao Yang, Penghui Wang, Haonan Liu, Jiaheng Zhao, Yongchi Feng, Xiachong Mao, Xin Yeung, Man Tsung Pipatanakul, Kunat Koto, Fajri Thu, Min Si Kydlíček, Hynek Liu, Zeyi Lin, Qunshu Sripaisarnmongkol, Sittipong Sae-Khow, Kridtaphad Thongchim, Nirattisai Konkaew, Taechawat Borijindargoon, Narong Dao, Anh Maneegard, Matichon Artkaew, Phakphum Yong, Zheng-Xin Nguyen, Quan Phatthiyaphaibun, Wannaphong Tran, Hoang H. Zhang, Mike Chen, Shiqi Pang, Tianyu Du, Chao Wan, Xinyi Lu, Wei Lin, Min Computation and Language Artificial Intelligence Machine Learning Sailor2 is a family of cutting-edge multilingual language models for South-East Asian (SEA) languages, available in 1B, 8B, and 20B sizes to suit diverse applications. Building on Qwen2.5, Sailor2 undergoes continuous pre-training on 500B tokens (400B SEA-specific and 100B replay tokens) to support 13 SEA languages while retaining proficiency in Chinese and English. Sailor2-20B model achieves a 50-50 win rate against GPT-4o across SEA languages. We also deliver a comprehensive cookbook on how to develop the multilingual model in an efficient manner, including five key aspects: data curation, pre-training, post-training, model customization and evaluation. We hope that Sailor2 model (Apache 2.0 license) will drive language development in the SEA region, and Sailor2 cookbook will inspire researchers to build more inclusive LLMs for other under-served languages.
title	Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
topic	Computation and Language Artificial Intelligence Machine Learning
url	https://arxiv.org/abs/2502.12982