Imagen de Portada

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Lujun, Qiyuan, Zhu, Wang, Jiacheng, Li, Wei, Gu, Hao, Han, Sirui, Guo, Yike
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Delta Decompression for MoE-based LLMs Compression
por: Gu, Hao, et al.
Publicado: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
por: Chen, Xiaodong, et al.
Publicado: (2025)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
por: Miao, Ruijie, et al.
Publicado: (2025)

BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
por: Gu, Hao, et al.
Publicado: (2025)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
por: Jin, Peng, et al.
Publicado: (2024)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
por: Neogi, Pinaki Prasad Guha, et al.
Publicado: (2025)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
por: Wang, Wenfeng, et al.
Publicado: (2025)

RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
por: Zhong, Zhengjia, et al.
Publicado: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
por: Gu, Yupu, et al.
Publicado: (2026)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
por: Liu, Xinyi, et al.
Publicado: (2026)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
por: Takashiro, Shota, et al.
Publicado: (2026)

Horseshoe Mixtures-of-Experts (HS-MoE)
por: Polson, Nick, et al.
Publicado: (2026)

Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
por: Xu, Binxing, et al.
Publicado: (2026)

VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
por: Chen, Hao, et al.
Publicado: (2024)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
por: Zhao, Yushu, et al.
Publicado: (2025)

PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
por: Zhang, Ze Yu, et al.
Publicado: (2025)

DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models
por: Aghdam, Maryam Akhavan, et al.
Publicado: (2024)

VisMMOE: Exploiting Visual-Expert Affinity for Efficient Visual-Language MoE Offloading
por: Xu, Cheng, et al.
Publicado: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
por: Xia, Xinfeng, et al.
Publicado: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
por: Hao, Jiawei, et al.
Publicado: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
por: Muzio, Alexandre, et al.
Publicado: (2024)

Expert Routing for Communication-Efficient MoE via Finite Expert Banks
por: Salehi, Mohammad Reza Deylam, et al.
Publicado: (2026)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
por: Yang, Cheng, et al.
Publicado: (2024)

Steering MoE LLMs via Expert (De)Activation
por: Fayyaz, Mohsen, et al.
Publicado: (2025)

Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference
por: Han, Ziyi, et al.
Publicado: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
por: Ma, Songkai, et al.
Publicado: (2025)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
por: Li, Xurui, et al.
Publicado: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
por: Gu, Naibin, et al.
Publicado: (2025)

MoE-Spec: Expert Budgeting for Efficient Speculative Decoding
por: McDanel, Bradley, et al.
Publicado: (2026)

Mixture of Experts (MoE): A Big Data Perspective
por: Gan, Wensheng, et al.
Publicado: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
por: Kulibaba, Stepan, et al.
Publicado: (2026)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
por: Ludziejewski, Jan, et al.
Publicado: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
por: Pióro, Maciej, et al.
Publicado: (2024)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
por: Li, Zichong, et al.
Publicado: (2025)

FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
por: Zhang, Boyang, et al.
Publicado: (2025)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
por: Sun, Libo, et al.
Publicado: (2026)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
por: Sun, Weigao, et al.
Publicado: (2025)

ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration
por: Ai, Mengting, et al.
Publicado: (2025)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
por: Chen, Yuanteng, et al.
Publicado: (2025)