Cover Image

AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Haiquan, Zhou, Yefan, Liu, Shiwei, Wang, Zhangyang, Mahoney, Michael W., Yang, Yaoqing
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Three-regime Model of Network Pruning
by: Zhou, Yefan, et al.
Published: (2023)

AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
by: He, Di, et al.
Published: (2025)

AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality
by: Qing, Peijun, et al.
Published: (2024)

Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
by: Lu, Haiquan, et al.
Published: (2024)

GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching
by: Su, Guinan, et al.
Published: (2025)

A Model Zoo on Phase Transitions in Neural Networks
by: Schürholt, Konstantin, et al.
Published: (2025)

SlimGPT: Layer-wise Structured Pruning for Large Language Models
by: Ling, Gui, et al.
Published: (2024)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

Models of Heavy-Tailed Mechanistic Universality
by: Hodgkinson, Liam, et al.
Published: (2025)

Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
by: Yin, Lu, et al.
Published: (2023)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
by: Cao, Mingyu, et al.
Published: (2024)

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction
by: Pang, Tianyu, et al.
Published: (2026)

Dynamic Encoder Size Based on Data-Driven Layer-wise Pruning for Speech Recognition
by: Xu, Jingjing, et al.
Published: (2024)

CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models
by: Tang, Zicong, et al.
Published: (2025)

Straightforward Layer-wise Pruning for More Efficient Visual Adaptation
by: Han, Ruizi, et al.
Published: (2024)

MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric
by: Lin, Haokun, et al.
Published: (2024)

Topology-Aware Layer Pruning for Large Vision-Language Models
by: Zheng, Pengcheng, et al.
Published: (2026)

The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Models
by: Lu, Yao, et al.
Published: (2025)

LaCo: Large Language Model Pruning via Layer Collapse
by: Yang, Yifei, et al.
Published: (2024)

From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
by: Kothapalli, Vignesh, et al.
Published: (2024)

IG-Pruning: Input-Guided Block Pruning for Large Language Models
by: Qiao, Kangyu, et al.
Published: (2025)

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models
by: Kang, Yuhan, et al.
Published: (2025)

PruneCD: Contrasting Pruned Self Model to Improve Decoding Factuality
by: Yu, Byeongho, et al.
Published: (2025)

PruneVid: Visual Token Pruning for Efficient Video Large Language Models
by: Huang, Xiaohu, et al.
Published: (2024)

Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression
by: Ilin, Ivan, et al.
Published: (2025)

FGP: Feature-Gradient-Prune for Efficient Convolutional Layer Pruning
by: Lv, Qingsong, et al.
Published: (2024)

When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs
by: Wang, Keyu, et al.
Published: (2025)

MD tree: a model-diagnostic tree grown on loss landscape
by: Zhou, Yefan, et al.
Published: (2024)

EntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models
by: Wang, Yahong, et al.
Published: (2026)

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models
by: Shrestha, Safal, et al.
Published: (2026)

SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning
by: Wang, Hanzhen, et al.
Published: (2025)

Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR
by: Irigoyen, Julian, et al.
Published: (2025)

A Simple Linear Patch Revives Layer-Pruned Large Language Models
by: Chen, Xinrui, et al.
Published: (2025)

Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
by: Bandari, Abhinav, et al.
Published: (2024)

Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Models
by: Xiang, Yang, et al.
Published: (2025)

Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models
by: Wang, Ziyan, et al.
Published: (2025)

FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models
by: Zhang, Yang, et al.
Published: (2024)

Learning to Discover Iterative Spectral Algorithms
by: Liu, Zihang, et al.
Published: (2026)

Large Language Model Pruning
by: Huang, Hanjuan, et al.
Published: (2024)

Prune, Update and Trim: Robust Structured Pruning for Large Language Models
by: Mecke, Diego Coello de Portugal, et al.
Published: (2026)