Copertina

EMO: Frustratingly Easy Progressive Training of Extendable MoE

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jin, Linghao, Shi, Chufan, Wang, Huijuan, Wen, Nuan, Liu, Zhengzhong, Xing, Eric, Ma, Xuezhe
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Towards Chapter-to-Chapter Context-Aware Literary Translation via Large Language Models
di: Jin, Linghao, et al.
Pubblicazione: (2024)

Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models
di: Han, Xu, et al.
Pubblicazione: (2024)

Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communities
di: Wen, Nuan, et al.
Pubblicazione: (2026)

GQA-μP: The maximal parameterization update for grouped query attention
di: Chickering, Kyle R., et al.
Pubblicazione: (2026)

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
di: Shi, Chufan, et al.
Pubblicazione: (2024)

Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning
di: Wang, Ximei, et al.
Pubblicazione: (2023)

ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
di: Jin, Chao, et al.
Pubblicazione: (2026)

HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap
di: Lin, Wenxiang, et al.
Pubblicazione: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
di: Ma, Songkai, et al.
Pubblicazione: (2025)

MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing
di: Ma, Haiyue, et al.
Pubblicazione: (2025)

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
di: Shi, Xiaoming, et al.
Pubblicazione: (2024)

PR2: Predictive Routing Replay for MoE-Based LLM Reinforcement Learning
di: Dong, Daize, et al.
Pubblicazione: (2026)

Muon: Training and Trade-offs with Latent Attention and MoE
di: Mehta, Sushant, et al.
Pubblicazione: (2025)

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
di: Xu, Zukang, et al.
Pubblicazione: (2026)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
di: Jin, Chao, et al.
Pubblicazione: (2025)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
di: Ma, Wenhan, et al.
Pubblicazione: (2025)

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
di: Wu, Haoze, et al.
Pubblicazione: (2024)

Grouter: Decoupling Routing from Representation for Accelerated MoE Training
di: Xu, Yuqi, et al.
Pubblicazione: (2026)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
di: Liu, Dennis, et al.
Pubblicazione: (2025)

Mol-MoE: Training Preference-Guided Routers for Molecule Generation
di: Calanzone, Diego, et al.
Pubblicazione: (2025)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
di: Hannah, Lauren. A, et al.
Pubblicazione: (2025)

Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training
di: Du, Xianzhi, et al.
Pubblicazione: (2024)

GRIN: GRadient-INformed MoE
di: Liu, Liyuan, et al.
Pubblicazione: (2024)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
di: Liu, Qingxiu, et al.
Pubblicazione: (2026)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
di: Miao, Ruijie, et al.
Pubblicazione: (2025)

AIMER: Calibration-Free Task-Agnostic MoE Pruning
di: Liu, Zongfang, et al.
Pubblicazione: (2026)

Asymmetric Idiosyncrasies in Multimodal Models
di: Tao, Muzi, et al.
Pubblicazione: (2026)

Frustratingly Easy Task-aware Pruning for Large Language Models
di: Tian, Yuanhe, et al.
Pubblicazione: (2025)

MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
di: Xue, Leyang, et al.
Pubblicazione: (2024)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
di: Manzoni, Andrea
Pubblicazione: (2026)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
di: Cao, Shiyi, et al.
Pubblicazione: (2024)

Llama 3 Meets MoE: Efficient Upcycling
di: Vavre, Aditya, et al.
Pubblicazione: (2024)

DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models
di: Aghdam, Maryam Akhavan, et al.
Pubblicazione: (2024)

Faster MoE LLM Inference for Extremely Large Models
di: Yang, Haoqi, et al.
Pubblicazione: (2025)

Practical FP4 Training for Large-Scale MoE Models on Hopper GPUs
di: Zhang, Wuyue, et al.
Pubblicazione: (2026)

Horseshoe Mixtures-of-Experts (HS-MoE)
di: Polson, Nick, et al.
Pubblicazione: (2026)

LocMoE: A Low-Overhead MoE for Large Language Model Training
di: Li, Jing, et al.
Pubblicazione: (2024)

MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs
di: Zhang, Jiyuan, et al.
Pubblicazione: (2026)

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
di: Zhao, Jiayu, et al.
Pubblicazione: (2026)