Cover Image

Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lucas, Ryan, Mazumder, Rahul
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models
by: Meng, Xiang, et al.
Published: (2024)

OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
by: Meng, Xiang, et al.
Published: (2024)

ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
by: Lucas, Ryan, et al.
Published: (2026)

FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning
by: Meng, Xiang, et al.
Published: (2024)

FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML
by: Liu, Brian, et al.
Published: (2024)

MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
by: Afriat, Gabriel, et al.
Published: (2026)

Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
by: Zhao, Yanjun, et al.
Published: (2024)

Second-Order Min-Max Optimization with Lazy Hessians
by: Chen, Lesi, et al.
Published: (2024)

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models
by: Kang, Yuhan, et al.
Published: (2025)

MOSS: Multi-Objective Optimization for Stable Rule Sets
by: Liu, Brian, et al.
Published: (2025)

Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests
by: Liu, Brian, et al.
Published: (2024)

3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
by: Makni, Mehdi, et al.
Published: (2026)

Sparse Gaussian Graphical Models with Discrete Optimization: Computational and Statistical Perspectives
by: Behdin, Kayhan, et al.
Published: (2023)

CORP: Closed-Form One-shot Representation-Preserving Structured Pruning for Transformers
by: Zhang, Boxiang, et al.
Published: (2026)

GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning
by: Hamidi, Shayan Mohajer, et al.
Published: (2024)

Sparse NMF with Archetypal Regularization: Computational and Robustness Properties
by: Behdin, Kayhan, et al.
Published: (2021)

Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing
by: Romanov, Elad, et al.
Published: (2024)

Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
by: Manivannan, Sanjeev, et al.
Published: (2026)

Fast Deep Hedging with Second-Order Optimization
by: Mueller, Konrad, et al.
Published: (2024)

Subspace-based Approximate Hessian Method for Zeroth-Order Optimization
by: Kim, Dongyoon, et al.
Published: (2025)

Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
by: Qiu, Junbin, et al.
Published: (2026)

Hess-MC2: Sequential Monte Carlo Squared using Hessian Information and Second Order Proposals
by: Murphy, Joshua, et al.
Published: (2025)

Beyond Manually Designed Pruning Policies with Second-Level Performance Prediction: A Pruning Framework for LLMs
by: Ma, Zuxin, et al.
Published: (2025)

An Optimization Framework for Differentially Private Sparse Fine-Tuning
by: Makni, Mehdi, et al.
Published: (2025)

SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
by: Shin, Dahun, et al.
Published: (2025)

SGD with Partial Hessian for Deep Neural Networks Optimization
by: Sun, Ying, et al.
Published: (2024)

HASSLE-free: A unified Framework for Sparse plus Low-Rank Matrix Decomposition for LLMs
by: Makni, Mehdi, et al.
Published: (2025)

Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimization
by: Huang, Feihu
Published: (2024)

Extracting Interpretable Models from Tree Ensembles: Computational and Statistical Perspectives
by: Liu, Brian, et al.
Published: (2025)

End-to-end Feature Selection Approach for Learning Skinny Trees
by: Ibrahim, Shibal, et al.
Published: (2023)

Signal Collapse in One-Shot Pruning: When Sparse Models Fail to Distinguish Neural Representations
by: Saikumar, Dhananjay, et al.
Published: (2025)

One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
by: Janusz, Mikołaj, et al.
Published: (2025)

Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks
by: Yue, Yanwei, et al.
Published: (2024)

Second Order Methods for Bandit Optimization and Control
by: Suggala, Arun, et al.
Published: (2024)

FOSI: Hybrid First and Second Order Optimization
by: Sivan, Hadar, et al.
Published: (2023)

SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shot
by: Tuo, Kaiwen, et al.
Published: (2025)

Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits
by: Sadiev, Abdurakhmon, et al.
Published: (2025)

HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Space
by: Li, Ke, et al.
Published: (2025)

Second-Order Forward-Mode Automatic Differentiation for Optimization
by: Cobb, Adam D., et al.
Published: (2024)

Hessian-Free Online Certified Unlearning
by: Qiao, Xinbao, et al.
Published: (2024)