Image de couverture de livre

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Jin, Peng, Zhu, Bo, Yuan, Li, Yan, Shuicheng
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
par: Su, Yang, et autres
Publié: (2025)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
par: Yang, Cheng, et autres
Publié: (2024)

MoH: Multi-Head Attention as Mixture-of-Head Attention
par: Jin, Peng, et autres
Publié: (2024)

Mixture of Experts (MoE): A Big Data Perspective
par: Gan, Wensheng, et autres
Publié: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
par: Kulibaba, Stepan, et autres
Publié: (2026)

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
par: Shi, Xiaoming, et autres
Publié: (2024)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
par: Chen, Yuanteng, et autres
Publié: (2025)

Symphony-MoE: Harmonizing Disparate Pre-trained Models into a Coherent Mixture-of-Experts
par: Wang, Qi, et autres
Publié: (2025)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
par: Gu, Naibin, et autres
Publié: (2025)

Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts
par: Yun, Sukwon, et autres
Publié: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

Expert Divergence Learning for MoE-based Language Models
par: Li, Jiaang, et autres
Publié: (2026)

MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
par: Tran, Huyen Ngoc, et autres
Publié: (2026)

BrainNet-MoE: Brain-Inspired Mixture-of-Experts Learning for Neurological Disease Identification
par: Zhang, Jing, et autres
Publié: (2025)

Accelerating MoE Model Inference with Expert Sharding
par: Balmau, Oana, et autres
Publié: (2025)

MoE-DisCo:Low Economy Cost Training Mixture-of-Experts Models
par: Ye, Xin, et autres
Publié: (2026)

MoE-Health: A Mixture of Experts Framework for Robust Multimodal Healthcare Prediction
par: Wang, Xiaoyang, et autres
Publié: (2025)

Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
par: Ye, Charles, et autres
Publié: (2026)

BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference
par: Wang, Yun, et autres
Publié: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
par: Li, Pingzhi, et autres
Publié: (2025)

SD-MoE: Spectral Decomposition for Effective Expert Specialization
par: Huang, Ruijun, et autres
Publié: (2026)

I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts
par: Xin, Jiayi, et autres
Publié: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
par: Pióro, Maciej, et autres
Publié: (2024)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
par: Ludziejewski, Jan, et autres
Publié: (2025)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
par: Koike-Akino, Toshiaki, et autres
Publié: (2025)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
par: Neogi, Pinaki Prasad Guha, et autres
Publié: (2025)

DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts
par: Yao, Zelin, et autres
Publié: (2024)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
par: Sun, Weigao, et autres
Publié: (2025)

RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
par: Zhong, Zhengjia, et autres
Publié: (2026)

Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
par: Hua, Yongxiang, et autres
Publié: (2025)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
par: Kunwar, Pradip, et autres
Publié: (2025)

L-MoE: End-to-End Training of a Lightweight Mixture of Low-Rank Adaptation Experts
par: Ji, Shihao, et autres
Publié: (2025)

Seg-MoE: Multi-Resolution Segment-wise Mixture-of-Experts for Time Series Forecasting Transformers
par: Ortigossa, Evandro S., et autres
Publié: (2026)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
par: Yan, Jiaming, et autres
Publié: (2025)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
par: Huang, Wei, et autres
Publié: (2025)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
par: Li, Xurui, et autres
Publié: (2024)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
par: Hao, Jiawei, et autres
Publié: (2026)

Awakening Dormant Experts:Counterfactual Routing to Mitigate MoE Hallucinations
par: Hu, Wentao, et autres
Publié: (2026)

Speculating Experts Accelerates Inference for Mixture-of-Experts
par: Madan, Vivan, et autres
Publié: (2026)

SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models
par: Wu, Juntong, et autres
Publié: (2026)