Cover Image

Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Di, Wu, Xun, Huang, Shaohan, Jiang, Lingjie, Hao, Yaru, Dong, Li, Chi, Zewen, Sui, Zhifang, Wei, Furu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

On-Policy RL with Optimal Reward Baseline
by: Hao, Yaru, et al.
Published: (2025)

Multi-Head Mixture-of-Experts
by: Wu, Xun, et al.
Published: (2024)

Textual Aesthetics in Large Language Models
by: Jiang, Lingjie, et al.
Published: (2024)

Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
by: Li, Zongqian, et al.
Published: (2026)

The Era of Agentic Organization: Learning to Organize with Language Models
by: Chi, Zewen, et al.
Published: (2025)

MH-MoE: Multi-Head Mixture-of-Experts
by: Huang, Shaohan, et al.
Published: (2024)

Black-Box On-Policy Distillation of Large Language Models
by: Ye, Tianzhu, et al.
Published: (2025)

Think Only When You Need with Large Hybrid-Reasoning Models
by: Jiang, Lingjie, et al.
Published: (2025)

Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
by: Zhang, Di, et al.
Published: (2026)

Optimizing Prompts for Text-to-Image Generation
by: Hao, Yaru, et al.
Published: (2022)

Reward Reasoning Model
by: Guo, Jiaxin, et al.
Published: (2025)

VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
by: Jiang, Lingjie, et al.
Published: (2025)

BitNet Distillation
by: Wu, Xun, et al.
Published: (2025)

Online Experiential Learning for Language Models
by: Ye, Tianzhu, et al.
Published: (2026)

Code Aesthetics with Agentic Reward Feedback
by: Xiao, Bang, et al.
Published: (2025)

Thinking Augmented Pre-training
by: Wang, Liang, et al.
Published: (2025)

On-Policy Context Distillation for Language Models
by: Ye, Tianzhu, et al.
Published: (2026)

Towards Optimal Learning of Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Reinforcement Pre-Training
by: Dong, Qingxiu, et al.
Published: (2025)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

Maximum Score Routing For Mixture-of-Experts
by: Dong, Bowen, et al.
Published: (2025)

Text Diffusion with Reinforced Conditioning
by: Liu, Yuxuan, et al.
Published: (2024)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

Adapting Large Language Models to Domains via Reading Comprehension
by: Cheng, Daixuan, et al.
Published: (2023)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
by: Kim, Gyeongman, et al.
Published: (2025)

Towards a Comprehensive Scaling Law of Mixture-of-Experts
by: Zhao, Guoliang, et al.
Published: (2025)

Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems
by: Li, Zongqian, et al.
Published: (2026)

SlideSparse: Fast and Flexible (2N-2):2N Structured Sparsity
by: Shao, Hanyong, et al.
Published: (2026)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
by: Huang, Wei, et al.
Published: (2024)

Fleming-R1: Toward Expert-Level Medical Reasoning via Reinforcement Learning
by: Liu, Chi, et al.
Published: (2025)

Exploring Activation Patterns of Parameters in Language Models
by: Wang, Yudong, et al.
Published: (2024)

Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation
by: Wu, Xun, et al.
Published: (2024)

Towards Principled Design of Mixture-of-Experts Language Models under Memory and Inference Constraints
by: Liew, Seng Pei, et al.
Published: (2026)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
by: Zheng, Chen, et al.
Published: (2025)

BitNet b1.58 2B4T Technical Report
by: Ma, Shuming, et al.
Published: (2025)

Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs
by: Wang, Qibin, et al.
Published: (2025)

A Survey on Mixture of Experts in Large Language Models
by: Cai, Weilin, et al.
Published: (2024)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
by: Ma, Wenhan, et al.
Published: (2025)

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
by: Dai, Damai, et al.
Published: (2024)