Cover Image

Prune&Comp: Free Lunch for Layer-Pruned LLMs via Iterative Pruning with Magnitude Compensation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Xinrui, Zhang, Hongxing, Zeng, Fanyi, Wei, Yongxian, Wang, Yizhi, Ling, Xitong, Li, Guanghao, Yuan, Chun
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Iterative Layer Pruning for Efficient Translation Inference
by: Moslem, Yasmin, et al.
Published: (2025)

A Simple Linear Patch Revives Layer-Pruned Large Language Models
by: Chen, Xinrui, et al.
Published: (2025)

GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensation
by: Liu, Hao, et al.
Published: (2026)

ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
by: Hou, Bairu, et al.
Published: (2025)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
by: Fu, Yao, et al.
Published: (2025)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
by: Le, Qi, et al.
Published: (2025)

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
by: Kolawole, Steven, et al.
Published: (2024)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
by: Cao, Mingyu, et al.
Published: (2024)

On Pruning State-Space LLMs
by: Ghattas, Tamer, et al.
Published: (2025)

Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
by: Liu, Deyuan, et al.
Published: (2024)

From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
by: Pei, Zehua, et al.
Published: (2024)

Optimizing Korean-Centric LLMs via Token Pruning
by: Kim, Hoyeol, et al.
Published: (2026)

High-Layer Attention Pruning with Rescaling
by: Liu, Songtao, et al.
Published: (2025)

Prune as You Generate: Online Rollout Pruning for Faster and Better RLVR
by: Xu, Haobo, et al.
Published: (2026)

IG-Pruning: Input-Guided Block Pruning for Large Language Models
by: Qiao, Kangyu, et al.
Published: (2025)

DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
by: Gu, Tianteng, et al.
Published: (2025)

Less is More for RAG: Information Gain Pruning for Generator-Aligned Reranking and Evidence Selection
by: Song, Zhipeng, et al.
Published: (2026)

Pruning Strategies for Backdoor Defense in LLMs
by: Chapagain, Santosh, et al.
Published: (2025)

Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
by: Chen, Hanting, et al.
Published: (2025)

PIP: Perturbation-based Iterative Pruning for Large Language Models
by: Cao, Yi, et al.
Published: (2025)

Spiking Layer-Adaptive Magnitude-based Pruning
by: Wang, Junqiao, et al.
Published: (2026)

Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration
by: Wu, Guangxin, et al.
Published: (2026)

Insights into the Lottery Ticket Hypothesis and Iterative Magnitude Pruning
by: Saleem, Tausifa Jan, et al.
Published: (2024)

NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models
by: Li, Shengrui, et al.
Published: (2024)

EvoP: Robust LLM Inference via Evolutionary Pruning
by: Wu, Shangyu, et al.
Published: (2025)

VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
by: Goel, Raghavv, et al.
Published: (2025)

Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
by: Li, Jianwei, et al.
Published: (2023)

Prune4Web: DOM Tree Pruning Programming for Web Agent
by: Zhang, Jiayuan, et al.
Published: (2025)

GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs
by: Huang, Wei, et al.
Published: (2026)

Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
by: Bandari, Abhinav, et al.
Published: (2024)

SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval
by: Lin, Yueqian, et al.
Published: (2024)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
by: Lee, Jaeseong, et al.
Published: (2024)

RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Models
by: Haruta, Shuichiro, et al.
Published: (2025)

Towards Extreme Pruning of LLMs with Plug-and-Play Mixed Sparsity
by: Xu, Chi, et al.
Published: (2025)

Towards Threshold-Free KV Cache Pruning
by: Ni, Xuanfan, et al.
Published: (2025)

Pruning as a Domain-specific LLM Extractor
by: Zhang, Nan, et al.
Published: (2024)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
by: Yu, Tongzhou, et al.
Published: (2025)

PruneTIR: Inference-Time Tool Call Pruning for Effective yet Efficient Tool-Integrated Reasoning
by: Zhang, Luan, et al.
Published: (2026)

Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR
by: Irigoyen, Julian, et al.
Published: (2025)