Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2412.04003 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915049409347584 |
|---|---|
| author | Ming, Lingfeng Zeng, Bo Lyu, Chenyang Shi, Tianqi Zhao, Yu Yang, Xue Liu, Yefeng Wang, Yiyu Xu, Linlong Liu, Yangyang Zhao, Xiaohu Wang, Hao Liu, Heng Zhou, Hao Yin, Huifeng Shang, Zifu Li, Haijun Wang, Longyue Luo, Weihua Zhang, Kaifu |
| author_facet | Ming, Lingfeng Zeng, Bo Lyu, Chenyang Shi, Tianqi Zhao, Yu Yang, Xue Liu, Yefeng Wang, Yiyu Xu, Linlong Liu, Yangyang Zhao, Xiaohu Wang, Hao Liu, Heng Zhou, Hao Yin, Huifeng Shang, Zifu Li, Haijun Wang, Longyue Luo, Weihua Zhang, Kaifu |
| contents | Large Language Models (LLMs) have achieved remarkable progress in recent years; however, their excellent performance is still largely limited to major world languages, primarily English. Many LLMs continue to face challenges with multilingual tasks, especially when it comes to low-resource languages. To address this issue, we introduced Marco-LLM: Massive multilingual training for cross-lingual enhancement LLM. We have collected a substantial amount of multilingual data for several low-resource languages and conducted extensive continual pre-training using the Qwen2 models. This effort has resulted in a multilingual LLM named Marco-LLM. Through comprehensive evaluations on various multilingual benchmarks, including MMMLU, AGIEval, Belebele, Flores-200, XCOPA and many others, Marco-LLM has demonstrated substantial improvements over state-of-the-art LLMs. Furthermore, Marco-LLM achieved substantial enhancements in any-to-any machine translation tasks, showing the effectiveness of our multilingual LLM. Marco-LLM is a pioneering multilingual LLM designed to not only perform exceptionally well in multilingual tasks, including low-resource languages, but also maintain strong performance in English and other major languages, closing the performance gap between high- and low-resource language capabilities. By bridging languages, this effort demonstrates our dedication to ensuring LLMs work accurately across various languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04003 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Marco-LLM: Bridging Languages via Massive Multilingual Training for Cross-Lingual Enhancement Ming, Lingfeng Zeng, Bo Lyu, Chenyang Shi, Tianqi Zhao, Yu Yang, Xue Liu, Yefeng Wang, Yiyu Xu, Linlong Liu, Yangyang Zhao, Xiaohu Wang, Hao Liu, Heng Zhou, Hao Yin, Huifeng Shang, Zifu Li, Haijun Wang, Longyue Luo, Weihua Zhang, Kaifu Computation and Language Large Language Models (LLMs) have achieved remarkable progress in recent years; however, their excellent performance is still largely limited to major world languages, primarily English. Many LLMs continue to face challenges with multilingual tasks, especially when it comes to low-resource languages. To address this issue, we introduced Marco-LLM: Massive multilingual training for cross-lingual enhancement LLM. We have collected a substantial amount of multilingual data for several low-resource languages and conducted extensive continual pre-training using the Qwen2 models. This effort has resulted in a multilingual LLM named Marco-LLM. Through comprehensive evaluations on various multilingual benchmarks, including MMMLU, AGIEval, Belebele, Flores-200, XCOPA and many others, Marco-LLM has demonstrated substantial improvements over state-of-the-art LLMs. Furthermore, Marco-LLM achieved substantial enhancements in any-to-any machine translation tasks, showing the effectiveness of our multilingual LLM. Marco-LLM is a pioneering multilingual LLM designed to not only perform exceptionally well in multilingual tasks, including low-resource languages, but also maintain strong performance in English and other major languages, closing the performance gap between high- and low-resource language capabilities. By bridging languages, this effort demonstrates our dedication to ensuring LLMs work accurately across various languages. |
| title | Marco-LLM: Bridging Languages via Massive Multilingual Training for Cross-Lingual Enhancement |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.04003 |