Cover Image

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Jaeseong, hwang, seung-won, Qiao, Aurick, Campos, Daniel F, Yao, Zhewei, He, Yuxiong
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Gold-Switch: Training-Free Superposition of Slow- and Fast- Thinking LLMs
by: Lee, Jaeseong, et al.
Published: (2025)

OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
by: Lee, Jaeseong, et al.
Published: (2025)

SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation
by: Qiao, Aurick, et al.
Published: (2024)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
by: Cao, Mingyu, et al.
Published: (2024)

MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
by: Su, Zhaoyuan, et al.
Published: (2026)

Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences
by: Bekman, Stas, et al.
Published: (2025)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
by: Koike-Akino, Toshiaki, et al.
Published: (2025)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
by: Tang, Shengkun, et al.
Published: (2026)

MoE Pathfinder: Trajectory-driven Expert Pruning
by: Yang, Xican, et al.
Published: (2025)

MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
by: Zhang, Geng, et al.
Published: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
by: Xie, Yanyue, et al.
Published: (2024)

Inference Scaling for Bridging Retrieval and Augmented Generation
by: Lee, Youngwon, et al.
Published: (2024)

CORD: Balancing COnsistency and Rank Distillation for Robust Retrieval-Augmented Generation
by: Lee, Youngwon, et al.
Published: (2024)

AIMER: Calibration-Free Task-Agnostic MoE Pruning
by: Liu, Zongfang, et al.
Published: (2026)

SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
by: Liao, Huanxuan, et al.
Published: (2025)

Learning to Hint for Reinforcement Learning
by: Xia, Yu, et al.
Published: (2026)

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
by: Kolawole, Steven, et al.
Published: (2024)

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
by: Fu, Yao, et al.
Published: (2025)

EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
by: Liu, Zongfang, et al.
Published: (2026)

SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications
by: Oliaro, Gabriele, et al.
Published: (2024)

ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
by: Gao, Shangqian, et al.
Published: (2025)

Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models
by: Wang, Ziyan, et al.
Published: (2025)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
by: Manzoni, Andrea
Published: (2026)

Olica: Efficient Structured Pruning of Large Language Models without Retraining
by: He, Jiujun, et al.
Published: (2025)

Structured vs. Unstructured Pruning: An Exponential Gap
by: Ferre', Davide, et al.
Published: (2026)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
by: Gu, Naibin, et al.
Published: (2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
by: Li, Zichong, et al.
Published: (2025)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

Millions of States: Designing a Scalable MoE Architecture with RWKV-7 Meta-learner
by: Xiao, Liu, et al.
Published: (2025)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
by: Xia, Xinfeng, et al.
Published: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
by: Muzio, Alexandre, et al.
Published: (2024)

Structured Pruning for Diverse Best-of-N Reasoning Optimization
by: Nguyen, Hieu Trung, et al.
Published: (2025)

Deterministic Differentiable Structured Pruning for Large Language Models
by: Huang, Weiyu, et al.
Published: (2026)

Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning
by: Wang, Zhaoyang, et al.
Published: (2026)

GRIN: GRadient-INformed MoE
by: Liu, Liyuan, et al.
Published: (2024)

VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
by: Goel, Raghavv, et al.
Published: (2025)

DarwinLM: Evolutionary Structured Pruning of Large Language Models
by: Tang, Shengkun, et al.
Published: (2025)

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
by: Lasby, Mike, et al.
Published: (2025)

Demystifying When Pruning Works via Representation Hierarchies
by: He, Shwai, et al.
Published: (2026)

On Pruning State-Space LLMs
by: Ghattas, Tamer, et al.
Published: (2025)