Cover Image

SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zimmer, Max, Roux, Christophe, Wagner, Moritz, Hendrych, Deborah, Pokutta, Sebastian
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Free Lunch in LLM Compression: Revisiting Retraining after Pruning
by: Wagner, Moritz, et al.
Published: (2025)

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging
by: Zimmer, Max, et al.
Published: (2023)

On the Byzantine-Resilience of Distillation-Based Federated Learning
by: Roux, Christophe, et al.
Published: (2024)

The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning
by: Zimmer, Max, et al.
Published: (2026)

PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
by: Zimmer, Max, et al.
Published: (2023)

From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
by: Schiekiera, Louis, et al.
Published: (2026)

Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
by: Roux, Christophe, et al.
Published: (2025)

Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers
by: Pelleriti, Nico, et al.
Published: (2025)

Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation
by: Pauls, Jan, et al.
Published: (2025)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling
by: Halder, Indranil, et al.
Published: (2025)

Estimating Canopy Height at Scale
by: Pauls, Jan, et al.
Published: (2024)

Convex mixed-integer optimization with Frank-Wolfe methods
by: Hendrych, Deborah, et al.
Published: (2022)

Interpretability Guarantees with Merlin-Arthur Classifiers
by: Wäldchen, Stephan, et al.
Published: (2022)

Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement
by: Wang, Anyi, et al.
Published: (2025)

Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling
by: Monjur, Ocean, et al.
Published: (2026)

SAFE: Finding Sparse and Flat Minima to Improve Pruning
by: Lee, Dongyeop, et al.
Published: (2025)

Restructuring Tractable Probabilistic Circuits
by: Zhang, Honghua, et al.
Published: (2024)

GSE: Group-wise Sparse and Explainable Adversarial Attacks
by: Sadiku, Shpresim, et al.
Published: (2023)

Compression-aware Training of Neural Networks using Frank-Wolfe
by: Zimmer, Max, et al.
Published: (2022)

Neural Parameter Regression for Explicit Representations of PDE Solution Operators
by: Mundinger, Konrad, et al.
Published: (2024)

SparseJEPA: Sparse Representation Learning of Joint Embedding Predictive Architectures
by: Hartman, Max, et al.
Published: (2025)

Sparse, Efficient and Explainable Data Attribution with DualXDA
by: Yolcu, Galip Ümit, et al.
Published: (2024)

The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models
by: Xiao, Yang, et al.
Published: (2025)

Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
by: Choi, Moonseok, et al.
Published: (2023)

Tractable Representation Learning with Probabilistic Circuits
by: Braun, Steven, et al.
Published: (2025)

Tractable Uncertainty-Aware Meta-Learning
by: Park, Young-Jin, et al.
Published: (2022)

Continuous Mixtures of Tractable Probabilistic Models
by: Correia, Alvaro H. C., et al.
Published: (2022)

Improving Robustness In Sparse Autoencoders via Masked Regularization
by: Narayanaswamy, Vivek, et al.
Published: (2026)

OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition
by: Zhang, Stephen, et al.
Published: (2024)

Tractable Sharpness-Aware Learning of Probabilistic Circuits
by: Suresh, Hrithik, et al.
Published: (2025)

A Tractable Inference Perspective of Offline RL
by: Liu, Xuejie, et al.
Published: (2023)

On the Tractability of SHAP Explanations under Markovian Distributions
by: Marzouk, Reda, et al.
Published: (2024)

RAP: Runtime Adaptive Pruning for LLM Inference
by: Liu, Huanrong, et al.
Published: (2025)

ECHOSAT: Estimating Canopy Height Over Space And Time
by: Pauls, Jan, et al.
Published: (2026)

Implicit Riemannian Optimism with Applications to Min-Max Problems
by: Roux, Christophe, et al.
Published: (2025)

UniPruning: Unifying Local Metric and Global Feedback for Scalable Sparse LLMs
by: Ding, Yizhuo, et al.
Published: (2025)

When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RL
by: Wang, Youting, et al.
Published: (2026)

Trainable Dynamic Mask Sparse Attention
by: Shi, Jingze, et al.
Published: (2025)

Locality-Aware Redundancy Pruning for LLM Depth Compression
by: Yun, Vincent-Daniel, et al.
Published: (2026)