Imagen de Portada

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Hao, Jiawei, Hao, Zhiwei, Guo, Jianyuan, Shen, Li, Luo, Yong, Hu, Han, Zeng, Dan
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference
por: Wang, Yun, et al.
Publicado: (2025)

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
por: Zhao, Hao, et al.
Publicado: (2024)

FlashMoE: Reducing SSD I/O Bottlenecks via ML-Based Cache Replacement for Mixture-of-Experts Inference on Edge Devices
por: Kim, Byeongju, et al.
Publicado: (2026)

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
por: Zhao, Heng, et al.
Publicado: (2026)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
por: Jin, Peng, et al.
Publicado: (2024)

Mixture of Weak & Strong Experts on Graphs
por: Zeng, Hanqing, et al.
Publicado: (2023)

Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
por: Tang, Anke, et al.
Publicado: (2024)

AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
por: Zeng, Zihao, et al.
Publicado: (2024)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
por: Yang, Cheng, et al.
Publicado: (2024)

MoNTA: Accelerating Mixture-of-Experts Training with Network-Traffc-Aware Parallel Optimization
por: Guo, Jingming, et al.
Publicado: (2024)

WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting
por: Wu, Shunyu, et al.
Publicado: (2026)

DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts
por: Yao, Zelin, et al.
Publicado: (2024)

OrdMoE: Preference Alignment via Hierarchical Expert Group Ranking in Multimodal Mixture-of-Experts LLMs
por: Gao, Yuting, et al.
Publicado: (2025)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
por: Chen, Yuanteng, et al.
Publicado: (2025)

Mixture of Experts (MoE): A Big Data Perspective
por: Gan, Wensheng, et al.
Publicado: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
por: Kulibaba, Stepan, et al.
Publicado: (2026)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
por: Xie, Zhitian, et al.
Publicado: (2024)

MobileMoE: Scaling On-Device Mixture of Experts
por: Chen, Yanbei, et al.
Publicado: (2026)

MoEMeta: Mixture-of-Experts Meta Learning for Few-Shot Relational Learning
por: Wu, Han, et al.
Publicado: (2025)

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
por: Su, Yang, et al.
Publicado: (2025)

MoSE: Unveiling Structural Patterns in Graphs via Mixture of Subgraph Experts
por: Ye, Junda, et al.
Publicado: (2025)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
por: Gao, Yuting, et al.
Publicado: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

Spatial-Temporal Mixture-of-Graph-Experts for Multi-Type Crime Prediction
por: Wu, Ziyang, et al.
Publicado: (2024)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
por: Nikolic, Strahinja, et al.
Publicado: (2025)

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
por: Hao, Zhiwei, et al.
Publicado: (2025)

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
por: Tang, Anke, et al.
Publicado: (2024)

Mixture of Experts in Large Language Models
por: Zhang, Danyang, et al.
Publicado: (2025)

MoIN: Mixture of Introvert Experts to Upcycle an LLM
por: Tejankar, Ajinkya, et al.
Publicado: (2024)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
por: Chu, Kexin, et al.
Publicado: (2025)

MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
por: Hu, Xing, et al.
Publicado: (2025)

MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
por: Tran, Huyen Ngoc, et al.
Publicado: (2026)

GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts
por: Guo, Zihao, et al.
Publicado: (2024)

MoEUT: Mixture-of-Experts Universal Transformers
por: Csordás, Róbert, et al.
Publicado: (2024)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
por: Zhao, Yushu, et al.
Publicado: (2025)

DynaMoE: Dynamic Token-Level Expert Activation with Layer-Wise Adaptive Capacity for Mixture-of-Experts Neural Networks
por: Gülmez, Gökdeniz
Publicado: (2026)

MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
por: Liu, Yiwen, et al.
Publicado: (2025)

HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts
por: Fang, Zihan, et al.
Publicado: (2026)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
por: Gu, Naibin, et al.
Publicado: (2025)

Speculating Experts Accelerates Inference for Mixture-of-Experts
por: Madan, Vivan, et al.
Publicado: (2026)