Cover Image

Each Prompt Matters: Scaling Reinforcement Learning Without Wasting Rollouts on Hundred-Billion-Scale MoE

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Anxiang, Zhang, Haibo, Zhang, Hailing, Mo, Kaixiang, Yao, Liang, Hu, Ling, Zhang, Long, Liu, Shuman, Xie, Shuyi, Li, Yanshi, Chen, Yizhang, Sheng, Yuepeng, Huang, Yuwei, Xu, Zhaochen, Zhou, Zhiqiang, Liew, Ziqin
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Reliable Evaluation of Large Language Models for Multilingual and Multimodal E-Commerce Applications
by: Xie, Shuyi, et al.
Published: (2025)

ESPO: Entropy Importance Sampling Policy Optimization
by: Sheng, Yuepeng, et al.
Published: (2025)

Compass-Thinker-7B Technical Report
by: Zeng, Anxiang, et al.
Published: (2025)

Decomposing and Steering Functional Metacognition in Large Language Models
by: Li, Yanshi, et al.
Published: (2026)

Mid-Training of Large Language Models: A Survey
by: Mo, Kaixiang, et al.
Published: (2025)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
by: Shi, Jingze, et al.
Published: (2026)

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
by: Shi, Xiaoming, et al.
Published: (2024)

Scaling Pre-training to One Hundred Billion Data for Vision Language Models
by: Wang, Xiao, et al.
Published: (2025)

MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
by: Zhou, Zhuoshan, et al.
Published: (2026)

Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling
by: Wang, Xinglin, et al.
Published: (2026)

Neural Attributed Community Search at Billion Scale
by: Wang, Jianwei, et al.
Published: (2024)

Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
by: Li, Yunxin, et al.
Published: (2025)

AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
by: Zhang, Bo-Wen, et al.
Published: (2024)

SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
by: Liu, Bingshuai, et al.
Published: (2025)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
by: Wang, Liujianfu, et al.
Published: (2025)

GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation
by: Xiong, Tianwei, et al.
Published: (2025)

Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
by: Yu, Zhongkai, et al.
Published: (2025)

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
by: Li, Yunxin, et al.
Published: (2024)

Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance
by: Wei, Yujie, et al.
Published: (2025)

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
by: Sun, Xingwu, et al.
Published: (2024)

Practical FP4 Training for Large-Scale MoE Models on Hopper GPUs
by: Zhang, Wuyue, et al.
Published: (2026)

Billions-Scale Forecast Reconciliation
by: Wang, Tianyu, et al.
Published: (2026)

EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
by: Sun, Quan, et al.
Published: (2024)

Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Data
by: Zhang, Shijie
Published: (2026)

Collaborative Compression for Large-Scale MoE Deployment on Edge
by: Chen, Yixiao, et al.
Published: (2025)

Scaling Learned Image Compression Models up to 1 Billion
by: Li, Yuqi, et al.
Published: (2025)

Breaking the Storage-Compute Bottleneck in Billion-Scale ANNS: A GPU-Driven Asynchronous I/O Framework
by: Xiao, Yang, et al.
Published: (2025)

U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF
by: Song, Xingchen, et al.
Published: (2024)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
by: Liu, Dennis, et al.
Published: (2025)

Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling
by: Liu, Yong, et al.
Published: (2026)

Billion-Scale Graph Foundation Models
by: Bechler-Speicher, Maya, et al.
Published: (2026)

ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
by: Li, Haley, et al.
Published: (2026)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
by: Zeng, Zhichen, et al.
Published: (2025)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
by: Han, Yu, et al.
Published: (2025)

Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
by: Zhang, Ruifeng, et al.
Published: (2026)

Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs
by: Zhang, Qijun, et al.
Published: (2026)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
by: Zhu, Fengqi, et al.
Published: (2025)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
by: Jin, Chao, et al.
Published: (2025)

ProMoE: Fast MoE-based LLM Serving using Proactive Caching
by: Song, Xiaoniu, et al.
Published: (2024)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
by: Chen, Liangkun, et al.
Published: (2025)