E$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918213293441024 |
|---|---|
| author | Yuan, Tao Bai, Haoli Pan, Yinfei Cao, Xuyang Zhang, Tianyu Hou, Lu Hu, Ting Yu, Xianzhi |
| author_facet | Yuan, Tao Bai, Haoli Pan, Yinfei Cao, Xuyang Zhang, Tianyu Hou, Lu Hu, Ting Yu, Xianzhi |
| contents | With the increasing size of large language models, layer pruning has gained increased attention as a hardware-friendly approach for model compression. However, existing layer pruning methods struggle to simultaneously address key practical deployment challenges, including performance degradation, high training costs, and limited acceleration. To overcome these limitations, we propose \name, a task-\underline{E}ffective, training-\underline{E}conomical and inference-\underline{E}fficient layer pruning framework. \namespace introduces two key innovations: (1) a differentiable mask optimization method using a Gumbel-TopK sampler, enabling efficient and precise pruning mask search; and (2) an entropy-aware adaptive knowledge distillation strategy that enhances task performance. Extensive experiments over diverse model architectures and benchmarks demonstrate the superiority of our method over state-of-the-art approaches. Notably, \namespace achieves 96\% accuracy, a mere 0.8\% drop from the original model (96.8\%) on MATH-500 when pruning 25\% layers of Qwen3-32B, outperforming existing SOTA (95\%), with a 1.33$\times$ inference speedup by consuming merely 0.5B tokens (0.5\% of the post-training data volume). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_17205 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | E$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models Yuan, Tao Bai, Haoli Pan, Yinfei Cao, Xuyang Zhang, Tianyu Hou, Lu Hu, Ting Yu, Xianzhi Computation and Language With the increasing size of large language models, layer pruning has gained increased attention as a hardware-friendly approach for model compression. However, existing layer pruning methods struggle to simultaneously address key practical deployment challenges, including performance degradation, high training costs, and limited acceleration. To overcome these limitations, we propose \name, a task-\underline{E}ffective, training-\underline{E}conomical and inference-\underline{E}fficient layer pruning framework. \namespace introduces two key innovations: (1) a differentiable mask optimization method using a Gumbel-TopK sampler, enabling efficient and precise pruning mask search; and (2) an entropy-aware adaptive knowledge distillation strategy that enhances task performance. Extensive experiments over diverse model architectures and benchmarks demonstrate the superiority of our method over state-of-the-art approaches. Notably, \namespace achieves 96\% accuracy, a mere 0.8\% drop from the original model (96.8\%) on MATH-500 when pruning 25\% layers of Qwen3-32B, outperforming existing SOTA (95\%), with a 1.33$\times$ inference speedup by consuming merely 0.5B tokens (0.5\% of the post-training data volume). |
| title | E$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.17205 |