Image de couverture de livre

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Lujun, Qiyuan, Zhu, Wang, Jiacheng, Li, Wei, Gu, Hao, Han, Sirui, Guo, Yike
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Delta Decompression for MoE-based LLMs Compression
par: Gu, Hao, et autres
Publié: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
par: Chen, Xiaodong, et autres
Publié: (2025)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
par: Miao, Ruijie, et autres
Publié: (2025)

BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
par: Gu, Hao, et autres
Publié: (2025)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
par: Neogi, Pinaki Prasad Guha, et autres
Publié: (2025)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
par: Wang, Wenfeng, et autres
Publié: (2025)

RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
par: Zhong, Zhengjia, et autres
Publié: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
par: Gu, Yupu, et autres
Publié: (2026)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
par: Liu, Xinyi, et autres
Publié: (2026)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
par: Takashiro, Shota, et autres
Publié: (2026)

Horseshoe Mixtures-of-Experts (HS-MoE)
par: Polson, Nick, et autres
Publié: (2026)

Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
par: Xu, Binxing, et autres
Publié: (2026)

VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
par: Chen, Hao, et autres
Publié: (2024)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
par: Zhao, Yushu, et autres
Publié: (2025)

PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
par: Zhang, Ze Yu, et autres
Publié: (2025)

DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models
par: Aghdam, Maryam Akhavan, et autres
Publié: (2024)

VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloading
par: Xu, Cheng, et autres
Publié: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
par: Xia, Xinfeng, et autres
Publié: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
par: Hao, Jiawei, et autres
Publié: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
par: Muzio, Alexandre, et autres
Publié: (2024)

Expert Routing for Communication-Efficient MoE via Finite Expert Banks
par: Salehi, Mohammad Reza Deylam, et autres
Publié: (2026)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
par: Yang, Cheng, et autres
Publié: (2024)

Steering MoE LLMs via Expert (De)Activation
par: Fayyaz, Mohsen, et autres
Publié: (2025)

Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference
par: Han, Ziyi, et autres
Publié: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
par: Ma, Songkai, et autres
Publié: (2025)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
par: Li, Xurui, et autres
Publié: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
par: Gu, Naibin, et autres
Publié: (2025)

MoE-Spec: Expert Budgeting for Efficient Speculative Decoding
par: McDanel, Bradley, et autres
Publié: (2026)

Mixture of Experts (MoE): A Big Data Perspective
par: Gan, Wensheng, et autres
Publié: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
par: Kulibaba, Stepan, et autres
Publié: (2026)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
par: Ludziejewski, Jan, et autres
Publié: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
par: Pióro, Maciej, et autres
Publié: (2024)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
par: Li, Zichong, et autres
Publié: (2025)

FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
par: Zhang, Boyang, et autres
Publié: (2025)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
par: Sun, Libo, et autres
Publié: (2026)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
par: Sun, Weigao, et autres
Publié: (2025)

ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
par: Ai, Mengting, et autres
Publié: (2025)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
par: Chen, Yuanteng, et autres
Publié: (2025)