Cover Image

Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Zihao, Ji, Shaoxiong, Luo, Hengyu, Tiedemann, Jörg
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data
by: Ji, Shaoxiong, et al.
Published: (2025)

A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives
by: Li, Zihao, et al.
Published: (2024)

Model-Based Quality Assessment for Massively Multilingual Parallel Data
by: Ibrahim, Abdelaziz M. A., et al.
Published: (2026)

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?
by: Ji, Shaoxiong, et al.
Published: (2024)

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models
by: Ji, Shaoxiong, et al.
Published: (2024)

Test-Time Scaling of Reasoning Models for Machine Translation
by: Li, Zihao, et al.
Published: (2025)

GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models
by: Luo, Hengyu, et al.
Published: (2025)

How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM
by: Ji, Shaoxiong, et al.
Published: (2024)

Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus
by: Joshi, Raviraj, et al.
Published: (2024)

MaLA-500: Massive Language Adaptation of Large Language Models
by: Lin, Peiqin, et al.
Published: (2024)

Scaling Low-Resource MT via Synthetic Data Generation with LLMs
by: de Gibert, Ona, et al.
Published: (2025)

Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters
by: Gurgurov, Daniil, et al.
Published: (2024)

The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
by: Shao, Jiandong, et al.
Published: (2026)

ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMs
by: Yoo, HanGyeol, et al.
Published: (2026)

Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning
by: Khade, Omkar, et al.
Published: (2024)

The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation
by: Itkonen, Oona, et al.
Published: (2026)

MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
by: Mickus, Timothee, et al.
Published: (2024)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
by: Ali, Mehdi, et al.
Published: (2025)

A New Massive Multilingual Dataset for High-Performance Language Technologies
by: de Gibert, Ona, et al.
Published: (2024)

Multilingual Language Model Pretraining using Machine-translated Data
by: Wang, Jiayi, et al.
Published: (2025)

HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models
by: Oepen, Stephan, et al.
Published: (2025)

PMC-InterCPT: Rethinking Biomedical Interleaved Data for Multimodal Continued Pretraining
by: Zhu, Guanghao, et al.
Published: (2026)

Toward Robust Multilingual Adaptation of LLMs for Low-Resource Languages
by: Li, Haolin, et al.
Published: (2025)

Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs
by: Zhou, Huichi, et al.
Published: (2025)

Framing Political Bias in Multilingual LLMs Across Pakistani Languages
by: Nadeem, Afrozah, et al.
Published: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
by: Foroutan, Negar, et al.
Published: (2025)

RedWhale: An Adapted Korean LLM Through Efficient Continual Pretraining
by: Vo, Anh-Dung, et al.
Published: (2024)

EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training
by: Dorkin, Aleksei, et al.
Published: (2026)

Multilingual Pretraining for Pixel Language Models
by: Kesen, Ilker, et al.
Published: (2025)

To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models
by: Domingo-Aldama, Ane G., et al.
Published: (2026)

Rethinking what Matters: Effective and Robust Multilingual Realignment for Low-Resource Languages
by: Nguyen, Quang Phuoc, et al.
Published: (2025)

Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets
by: Alrashed, Sultan, et al.
Published: (2025)

CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages
by: Yang, Yilun, et al.
Published: (2025)

Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval
by: Thakur, Nandan, et al.
Published: (2023)

Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages
by: Almheiri, Saeed, et al.
Published: (2026)

LangSAMP: Language-Script Aware Multilingual Pretraining
by: Liu, Yihong, et al.
Published: (2024)

Continual Pretraining on Encrypted Synthetic Data for Privacy-Preserving LLMs
by: Liu, Honghao, et al.
Published: (2026)

Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning
by: Li, Tong, et al.
Published: (2025)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
by: Ge, Ce, et al.
Published: (2024)

Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languages
by: Fong, Seraphina, et al.
Published: (2025)