Cover Image

A Free Lunch in LLM Compression: Revisiting Retraining after Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wagner, Moritz, Roux, Christophe, Zimmer, Max, Pokutta, Sebastian
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
by: Zimmer, Max, et al.
Published: (2025)

PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
by: Zimmer, Max, et al.
Published: (2023)

The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning
by: Zimmer, Max, et al.
Published: (2026)

On the Byzantine-Resilience of Distillation-Based Federated Learning
by: Roux, Christophe, et al.
Published: (2024)

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging
by: Zimmer, Max, et al.
Published: (2023)

From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
by: Schiekiera, Louis, et al.
Published: (2026)

Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
by: Roux, Christophe, et al.
Published: (2025)

No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
by: Zhang, Yanzhi, et al.
Published: (2025)

Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers
by: Pelleriti, Nico, et al.
Published: (2025)

Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression
by: Hyeon, Sieun, et al.
Published: (2026)

Compression-aware Training of Neural Networks using Frank-Wolfe
by: Zimmer, Max, et al.
Published: (2022)

Pruning Foundation Models for High Accuracy without Retraining
by: Zhao, Pu, et al.
Published: (2024)

Celo2: Towards Learned Optimization Free Lunch
by: Moudgil, Abhinav, et al.
Published: (2026)

Locality-Aware Redundancy Pruning for LLM Depth Compression
by: Yun, Vincent-Daniel, et al.
Published: (2026)

Interpretability Guarantees with Merlin-Arthur Classifiers
by: Wäldchen, Stephan, et al.
Published: (2022)

Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs
by: Li, Qingyuan, et al.
Published: (2024)

No Free Lunch: Fundamental Limits of Learning Non-Hallucinating Generative Models
by: Wu, Changlong, et al.
Published: (2024)

Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation
by: Pauls, Jan, et al.
Published: (2025)

Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling
by: Monjur, Ocean, et al.
Published: (2026)

A No Free Lunch Theorem for Human-AI Collaboration
by: Peng, Kenny, et al.
Published: (2024)

Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers
by: Hatefi, Sayed Mohammad Vakilzadeh, et al.
Published: (2024)

Improving Policy Optimization via $\varepsilon$-Retrain
by: Marzari, Luca, et al.
Published: (2024)

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models
by: Kang, Yuhan, et al.
Published: (2025)

Neural Parameter Regression for Explicit Representations of PDE Solution Operators
by: Mundinger, Konrad, et al.
Published: (2024)

CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
by: Li, Haoran, et al.
Published: (2026)

FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion
by: Wang, Zehan, et al.
Published: (2024)

Separable Power of Classical and Quantum Learning Protocols Through the Lens of No-Free-Lunch Theorem
by: Wang, Xinbiao, et al.
Published: (2024)

Estimating Canopy Height at Scale
by: Pauls, Jan, et al.
Published: (2024)

Assessing Per-Sample Membership Inference Vulnerability without Retraining
by: Dorseuil, Valentin, et al.
Published: (2026)

Free Lunch for Federated Remote Sensing Target Fine-Grained Classification: A Parameter-Efficient Framework
by: Chen, Shengchao, et al.
Published: (2024)

One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
by: Janusz, Mikołaj, et al.
Published: (2025)

RAP: KV-Cache Compression via RoPE-Aligned Pruning
by: Xin, Jihao, et al.
Published: (2026)

Free Lunch in Medical Image Foundation Model Pre-training via Randomized Synthesis and Disentanglement
by: Wei, Yuhan, et al.
Published: (2026)

RAP: Runtime Adaptive Pruning for LLM Inference
by: Liu, Huanrong, et al.
Published: (2025)

Data-Free Pruning of Self-Attention Layers in LLMs
by: Saikumar, Dhananjay, et al.
Published: (2025)

ECHOSAT: Estimating Canopy Height Over Space And Time
by: Pauls, Jan, et al.
Published: (2026)

Implicit Riemannian Optimism with Applications to Min-Max Problems
by: Roux, Christophe, et al.
Published: (2025)

Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
by: Abro, Aarash, et al.
Published: (2026)

No-Free-Lunch Theories for Tensor-Network Machine Learning Models
by: Wu, Jing-Chuan, et al.
Published: (2024)

LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation
by: Zhang, Weizhi, et al.
Published: (2025)