Image de couverture de livre

Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Hu, Wentao, Zhao, Mingkuan, Song, Shuangyong, Zhu, Xiaoyan, Lai, Xin, Wang, Jiayin
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Awakening Dormant Experts:Counterfactual Routing to Mitigate MoE Hallucinations
par: Hu, Wentao, et autres
Publié: (2026)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
par: Kim, Junhyuck, et autres
Publié: (2026)

Mosaic: Composite Projection Pruning for Resource-efficient LLMs
par: Eccles, Bailey J., et autres
Publié: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
par: Lu, Xudong, et autres
Publié: (2024)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
par: Xie, Yanyue, et autres
Publié: (2024)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
par: Yang, Cheng, et autres
Publié: (2024)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
par: Koike-Akino, Toshiaki, et autres
Publié: (2025)

Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesis
par: Li, Weikai, et autres
Publié: (2024)

DONOD: Efficient and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset Pruning
par: Hu, Jucheng, et autres
Publié: (2025)

StructPrune: Structured Global Pruning asymptotics with $\mathcal{O}(\sqrt{N})$ GPU Memory
par: Song, Xinyuan, et autres
Publié: (2025)

Numerical Pruning for Efficient Autoregressive Models
par: Shen, Xuan, et autres
Publié: (2024)

Sparsest Models Elude Pruning: An Exposé of Pruning's Current Capabilities
par: Zhang, Stephen, et autres
Publié: (2024)

Two-Stage Regularization-Based Structured Pruning for LLMs
par: Feng, Mingkuan, et autres
Publié: (2025)

HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Space
par: Li, Ke, et autres
Publié: (2025)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
par: Li, Yixiao, et autres
Publié: (2025)

Zero-shot Generalizable Graph Anomaly Detection with Mixture of Riemannian Experts
par: Zhao, Xinyu, et autres
Publié: (2026)

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models
par: Kang, Yuhan, et autres
Publié: (2025)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
par: Yu, Tongzhou, et autres
Publié: (2025)

Large Language Model Pruning
par: Huang, Hanjuan, et autres
Publié: (2024)

Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLMs
par: Ao, Shuang, et autres
Publié: (2025)

Mixture of Experts in Large Language Models
par: Zhang, Danyang, et autres
Publié: (2025)

REAM: Merging Improves Pruning of Experts in LLMs
par: Jha, Saurav, et autres
Publié: (2026)

Balanced Edge Pruning for Graph Anomaly Detection with Noisy Labels
par: Wang, Zhu, et autres
Publié: (2024)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
par: Le, Qi, et autres
Publié: (2025)

Hierarchical Attention-based Graph Neural Network with Relevance-driven Pruning
par: Kum, Seungwoo
Publié: (2026)

FCOS: A Two-Stage Recoverable Model Pruning Framework for Automatic Modulation Recognition
par: Lu, Yao, et autres
Publié: (2025)

Breaking Down Bias: On The Limits of Generalizable Pruning Strategies
par: Ma, Sibo, et autres
Publié: (2025)

Exploring Federated Pruning for Large Language Models
par: Guo, Pengxin, et autres
Publié: (2025)

Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models
par: Wang, Ziyan, et autres
Publié: (2025)

An Effective Information Theoretic Framework for Channel Pruning
par: Chen, Yihao, et autres
Publié: (2024)

Symmetric Pruning of Large Language Models
par: Yi, Kai, et autres
Publié: (2025)

Pruning for Generalization: A Transfer-Oriented Spatiotemporal Graph Framework
par: Jing, Zihao, et autres
Publié: (2026)

PruneSymNet: A Symbolic Neural Network and Pruning Algorithm for Symbolic Regression
par: Wu, Min, et autres
Publié: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
par: Gao, Yuting, et autres
Publié: (2025)

IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning
par: Jung, Jaeheun, et autres
Publié: (2025)

TopoPrune: Robust Data Pruning via Unified Latent Space Topology
par: Roy, Arjun, et autres
Publié: (2026)

CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction
par: Hu, Gang, et autres
Publié: (2026)

Meta Pruning via Graph Metanetworks : A Universal Meta Learning Framework for Network Pruning
par: Liu, Yewei, et autres
Publié: (2025)

Layer Collapse Can be Induced by Unstructured Pruning
par: Liao, Zhu, et autres
Publié: (2024)

MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models
par: Chamma, Ahmad, et autres
Publié: (2025)