Copertina

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Lujun, Qiyuan, Zhu, Wang, Jiacheng, Li, Wei, Gu, Hao, Han, Sirui, Guo, Yike
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Delta Decompression for MoE-based LLMs Compression
di: Gu, Hao, et al.
Pubblicazione: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
di: Chen, Xiaodong, et al.
Pubblicazione: (2025)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
di: Miao, Ruijie, et al.
Pubblicazione: (2025)

BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
di: Gu, Hao, et al.
Pubblicazione: (2025)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
di: Neogi, Pinaki Prasad Guha, et al.
Pubblicazione: (2025)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
di: Wang, Wenfeng, et al.
Pubblicazione: (2025)

RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
di: Zhong, Zhengjia, et al.
Pubblicazione: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)

MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
di: Gu, Yupu, et al.
Pubblicazione: (2026)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
di: Liu, Xinyi, et al.
Pubblicazione: (2026)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
di: Takashiro, Shota, et al.
Pubblicazione: (2026)

Horseshoe Mixtures-of-Experts (HS-MoE)
di: Polson, Nick, et al.
Pubblicazione: (2026)

Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
di: Xu, Binxing, et al.
Pubblicazione: (2026)

VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
di: Chen, Hao, et al.
Pubblicazione: (2024)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
di: Zhao, Yushu, et al.
Pubblicazione: (2025)

PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
di: Zhang, Ze Yu, et al.
Pubblicazione: (2025)

DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models
di: Aghdam, Maryam Akhavan, et al.
Pubblicazione: (2024)

VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloading
di: Xu, Cheng, et al.
Pubblicazione: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
di: Hao, Jiawei, et al.
Pubblicazione: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
di: Muzio, Alexandre, et al.
Pubblicazione: (2024)

Expert Routing for Communication-Efficient MoE via Finite Expert Banks
di: Salehi, Mohammad Reza Deylam, et al.
Pubblicazione: (2026)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
di: Yang, Cheng, et al.
Pubblicazione: (2024)

Steering MoE LLMs via Expert (De)Activation
di: Fayyaz, Mohsen, et al.
Pubblicazione: (2025)

Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference
di: Han, Ziyi, et al.
Pubblicazione: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
di: Ma, Songkai, et al.
Pubblicazione: (2025)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
di: Li, Xurui, et al.
Pubblicazione: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

MoE-Spec: Expert Budgeting for Efficient Speculative Decoding
di: McDanel, Bradley, et al.
Pubblicazione: (2026)

Mixture of Experts (MoE): A Big Data Perspective
di: Gan, Wensheng, et al.
Pubblicazione: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
di: Kulibaba, Stepan, et al.
Pubblicazione: (2026)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
di: Ludziejewski, Jan, et al.
Pubblicazione: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
di: Pióro, Maciej, et al.
Pubblicazione: (2024)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
di: Li, Zichong, et al.
Pubblicazione: (2025)

FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
di: Zhang, Boyang, et al.
Pubblicazione: (2025)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
di: Sun, Libo, et al.
Pubblicazione: (2026)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
di: Sun, Weigao, et al.
Pubblicazione: (2025)

ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
di: Ai, Mengting, et al.
Pubblicazione: (2025)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
di: Chen, Yuanteng, et al.
Pubblicazione: (2025)