Cover Image

A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$Δ$ Integration into Upcycled MoE

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Hao, Li, Tianhao, Wang, Zhijun, She, Shuaijie, Wu, Linjuan, Wei, Hao-Ran, Yang, Baosong, Chen, Jiajun, Huang, Shujian
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routing
by: Zhou, Hao, et al.
Published: (2024)

Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping
by: Zhu, Wenhao, et al.
Published: (2024)

How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective
by: Zhang, Shimao, et al.
Published: (2025)

Question Translation Training for Better Multilingual Reasoning
by: Zhu, Wenhao, et al.
Published: (2024)

MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
by: She, Shuaijie, et al.
Published: (2024)

Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
by: Jiang, Fan, et al.
Published: (2026)

Exploring the Factual Consistency in Dialogue Comprehension of Large Language Models
by: She, Shuaijie, et al.
Published: (2023)

MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting
by: Li, Tianhao, et al.
Published: (2024)

Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training
by: Wu, Linjuan, et al.
Published: (2025)

Llama 3 Meets MoE: Efficient Upcycling
by: Vavre, Aditya, et al.
Published: (2024)

R-PRM: Reasoning-Driven Process Reward Modeling
by: She, Shuaijie, et al.
Published: (2025)

Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions
by: Li, Jiahuan, et al.
Published: (2023)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
by: Wu, Haoyuan, et al.
Published: (2025)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
by: Li, Bo, et al.
Published: (2026)

Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling
by: Liao, Ning, et al.
Published: (2025)

Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs
by: Mirvakhabova, Leyla, et al.
Published: (2025)

Delta Decompression for MoE-based LLMs Compression
by: Gu, Hao, et al.
Published: (2025)

MoEless: Efficient MoE LLM Serving via Serverless Computing
by: Yu, Hanfei, et al.
Published: (2026)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
by: Li, Lujun, et al.
Published: (2025)

Trans-Zero: Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data
by: Zou, Wei, et al.
Published: (2025)

Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR
by: Yang, Hongli, et al.
Published: (2025)

EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE
by: Chen, Junyi, et al.
Published: (2023)

UMM-RM: An Upcycle-and-Merge MoE Reward Model for Mitigating Reward Hacking
by: Fu, Lingling, et al.
Published: (2025)

CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
by: Zhang, Jihai, et al.
Published: (2024)

Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training
by: Wang, Zhijun, et al.
Published: (2025)

Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
by: Luo, Jiajun, et al.
Published: (2024)

Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimation
by: Geng, Xiang, et al.
Published: (2025)

Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation
by: Zheng, Kening, et al.
Published: (2026)

MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism
by: Zhang, Zheng, et al.
Published: (2025)

STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation
by: Wang, Yiming, et al.
Published: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
by: Qian, Yulei, et al.
Published: (2024)

Language as a Latent Variable for Reasoning Optimization
by: Wu, Linjuan, et al.
Published: (2026)

Select-Mosaic: Data Augmentation Method for Dense Small Object Scenes
by: Zhang, Hao, et al.
Published: (2024)

Prescriptive Agents based on RAG for Automated Maintenance (PARAM)
by: Harbola, Chitranshu, et al.
Published: (2025)

Shape-IoU: More Accurate Metric considering Bounding Box Shape and Scale
by: Zhang, Hao, et al.
Published: (2023)

Focaler-IoU: More Focused Intersection over Union Loss
by: Zhang, Hao, et al.
Published: (2024)

TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoning
by: Huang, Xu, et al.
Published: (2026)

DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
by: Jin, Can, et al.
Published: (2025)

FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
by: Kang, Hao, et al.
Published: (2025)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
by: Zhu, Fengqi, et al.
Published: (2025)