Cover Image

Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zhijun, Li, Jiahuan, Zhou, Hao, Weng, Rongxiang, Wang, Jingang, Huang, Xin, Han, Xue, Feng, Junlan, Deng, Chao, Huang, Shujian
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routing
by: Zhou, Hao, et al.
Published: (2024)

Align to the Pivot: Dual Alignment with Self-Feedback for Multilingual Math Reasoning
by: Zhao, Chunxu, et al.
Published: (2026)

Self-Improving Multilingual Long Reasoning via Translation-Reasoning Integrated Training
by: Liu, Junxiao, et al.
Published: (2026)

Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners
by: Zhang, Shimao, et al.
Published: (2024)

Making Mathematical Reasoning Adaptive
by: Lai, Zhejian, et al.
Published: (2025)

Large Language Models Are Cross-Lingual Knowledge-Free Reasoners
by: Hu, Peng, et al.
Published: (2024)

PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning
by: Shen, Yunzhi, et al.
Published: (2026)

PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment
by: Li, Jiahuan, et al.
Published: (2024)

Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions
by: Li, Jiahuan, et al.
Published: (2023)

Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From
by: Gao, Changjiang, et al.
Published: (2025)

FIRE: Flexible Integration of Data Quality Ratings for Effective Pre-Training
by: Xu, Liangyu, et al.
Published: (2025)

Large-Scale Diverse Synthesis for Mid-Training
by: Zhang, Xuemiao, et al.
Published: (2025)

Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
by: Xu, Zhihao, et al.
Published: (2026)

GenDistiller: Distilling Pre-trained Language Models based on an Autoregressive Generative Model
by: Gao, Yingying, et al.
Published: (2024)

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
by: Fan, Yuchun, et al.
Published: (2026)

Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge
by: Li, Jiahuan, et al.
Published: (2024)

MT-PATCHER: Selective and Extendable Knowledge Distillation from Large Language Models for Machine Translation
by: Li, Jiahuan, et al.
Published: (2024)

SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset
by: Xie, Peng, et al.
Published: (2025)

A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$Δ$ Integration into Upcycled MoE
by: Zhou, Hao, et al.
Published: (2026)

A Survey on LLM Mid-Training
by: Tu, Chengying, et al.
Published: (2025)

Question Translation Training for Better Multilingual Reasoning
by: Zhu, Wenhao, et al.
Published: (2024)

MultiPL-MoE: Multi-Programming-Lingual Extension of Large Language Models through Hybrid Mixture-of-Experts
by: Wang, Qing, et al.
Published: (2025)

Palette of Language Models: A Solver for Controlled Text Generation
by: Yang, Zhe, et al.
Published: (2025)

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
by: Zheng, Qinkai, et al.
Published: (2023)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
by: Bai, Yang, et al.
Published: (2026)

SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment
by: Fan, Yuchun, et al.
Published: (2025)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoning
by: Huang, Xu, et al.
Published: (2026)

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
by: Zhang, Xuemiao, et al.
Published: (2025)

OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
by: Li, Han, et al.
Published: (2025)

PATS: Process-Level Adaptive Thinking Mode Switching
by: Wang, Yi, et al.
Published: (2025)

The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights
by: Zhu, Wenhao, et al.
Published: (2024)

Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-Learning
by: Han, Lifeng, et al.
Published: (2023)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
by: Zhang, Xuemiao, et al.
Published: (2025)

FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy
by: Zhang, Xuemiao, et al.
Published: (2025)

Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
by: Zhang, Xuemiao, et al.
Published: (2025)

Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping
by: Zhu, Wenhao, et al.
Published: (2024)

Depth Map Denoising Network and Lightweight Fusion Network for Enhanced 3D Face Recognition
by: Xu, Ruizhuo, et al.
Published: (2024)

BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models
by: Huang, Xu, et al.
Published: (2025)

Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment
by: Zhang, Jiazheng, et al.
Published: (2025)