Cover Image

ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Meng, Xiang, Behdin, Kayhan, Wang, Haoyue, Mazumder, Rahul
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
by: Meng, Xiang, et al.
Published: (2024)

Sparse NMF with Archetypal Regularization: Computational and Robustness Properties
by: Behdin, Kayhan, et al.
Published: (2021)

Sparse Gaussian Graphical Models with Discrete Optimization: Computational and Statistical Perspectives
by: Behdin, Kayhan, et al.
Published: (2023)

Differentially Private High-dimensional Variable Selection via Integer Programming
by: Prastakos, Petros, et al.
Published: (2025)

End-to-end Feature Selection Approach for Learning Skinny Trees
by: Ibrahim, Shibal, et al.
Published: (2023)

HASSLE-free: A unified Framework for Sparse plus Low-Rank Matrix Decomposition for LLMs
by: Makni, Mehdi, et al.
Published: (2025)

Sparse PCA: A New Scalable Estimator Based On Integer Programming
by: Behdin, Kayhan, et al.
Published: (2021)

Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints
by: Markovic-Voronov, Jelena, et al.
Published: (2026)

An Optimization Framework for Differentially Private Sparse Fine-Tuning
by: Makni, Mehdi, et al.
Published: (2025)

Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework
by: Lucas, Ryan, et al.
Published: (2024)

Multi-Task Learning for Sparsity Pattern Heterogeneity: Statistical and Computational Perspectives
by: Behdin, Kayhan, et al.
Published: (2022)

MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
by: Afriat, Gabriel, et al.
Published: (2026)

FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning
by: Meng, Xiang, et al.
Published: (2024)

TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks
by: Meng, Xiang, et al.
Published: (2025)

ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
by: Lucas, Ryan, et al.
Published: (2026)

3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
by: Makni, Mehdi, et al.
Published: (2026)

ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning
by: Su, Mingluo, et al.
Published: (2026)

Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction
by: Lucas, Ryan, et al.
Published: (2025)

Modeling with Categorical Features via Exact Fusion and Sparsity Regularisation
by: Behdin, Kayhan, et al.
Published: (2026)

SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shot
by: Tuo, Kaiwen, et al.
Published: (2025)

FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML
by: Liu, Brian, et al.
Published: (2024)

Computation of Least Trimmed Squares: A Branch-and-Bound framework with Hyperplane Arrangement Enhancements
by: Meng, Xiang, et al.
Published: (2026)

MOSS: Multi-Objective Optimization for Stable Rule Sets
by: Liu, Brian, et al.
Published: (2025)

BP-Seg: A graphical model approach to unsupervised and non-contiguous text segmentation using belief propagation
by: Li, Fengyi, et al.
Published: (2025)

Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism
by: Li, Guanchen, et al.
Published: (2024)

Signal Collapse in One-Shot Pruning: When Sparse Models Fail to Distinguish Neural Representations
by: Saikumar, Dhananjay, et al.
Published: (2025)

Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests
by: Liu, Brian, et al.
Published: (2024)

AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
by: Lu, Haiquan, et al.
Published: (2024)

SparseEval: Efficient Evaluation of Large Language Models by Sparse Optimization
by: Zhang, Taolin, et al.
Published: (2026)

Optimization-Inspired Few-Shot Adaptation for Large Language Models
by: Gao, Boyan, et al.
Published: (2025)

Extracting Interpretable Models from Tree Ensembles: Computational and Statistical Perspectives
by: Liu, Brian, et al.
Published: (2025)

One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
by: Janusz, Mikołaj, et al.
Published: (2025)

Route Sparse Autoencoder to Interpret Large Language Models
by: Shi, Wei, et al.
Published: (2025)

Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability
by: Liang, Tongtong, et al.
Published: (2026)

Pruning for Sparse Diffusion Models based on Gradient Flow
by: Wan, Ben, et al.
Published: (2025)

Prune, Update and Trim: Robust Structured Pruning for Large Language Models
by: Mecke, Diego Coello de Portugal, et al.
Published: (2026)

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging
by: Zimmer, Max, et al.
Published: (2023)

Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
by: Sui, Yueyuan, et al.
Published: (2026)

Beware of Calibration Data for Pruning Large Language Models
by: Ji, Yixin, et al.
Published: (2024)

Pruning is Optimal for Learning Sparse Features in High-Dimensions
by: Vural, Nuri Mert, et al.
Published: (2024)