Cover Image

Train Less, Infer Faster: Efficient Model Finetuning and Compression via Structured Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Svirsky, Jonathan, Refael, Yehonathan, Lindenbaum, Ofir
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FineGates: LLMs Finetuning with Compression using Stochastic Gates
by: Svirsky, Jonathan, et al.
Published: (2024)

AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning
by: Refael, Yehonathan, et al.
Published: (2024)

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
by: Arbel, Iftach, et al.
Published: (2024)

LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
by: Refael, Yehonathan, et al.
Published: (2025)

SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
by: Refael, Yehonathan, et al.
Published: (2025)

Interpretable Deep Clustering for Tabular Data
by: Svirsky, Jonathan, et al.
Published: (2023)

Unveiling Multiple Descents in Unsupervised Autoencoders
by: Rahimi, Kobi, et al.
Published: (2024)

Self Supervised Correlation-based Permutations for Multi-View Clustering
by: Eisenberg, Ran, et al.
Published: (2024)

Sparse Binarization for Fast Keyword Spotting
by: Svirsky, Jonathan, et al.
Published: (2024)

No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks
by: Refael, Yehonatan, et al.
Published: (2025)

Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization
by: Refael, Yehonathan, et al.
Published: (2022)

Learning Permutation from Structure Without Supervision
by: Eisenberg, Ran, et al.
Published: (2026)

Provable Speech Attributes Conversion via Latent Independence
by: Svirsky, Jonathan, et al.
Published: (2025)

Mathematical Framework for Online Social Media Auditing
by: Huleihel, Wasim, et al.
Published: (2022)

Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings
by: Naor, Erel, et al.
Published: (2025)

Uncovering a Winning Lottery Ticket with Continuously Relaxed Bernoulli Gates
by: Tsayag, Itamar, et al.
Published: (2026)

Conditional Deep Canonical Time Warping
by: Steinberg, Afek, et al.
Published: (2024)

Spectral Self-supervised Feature Selection
by: Segal, Daniel, et al.
Published: (2024)

Generalizable and Robust Spectral Method for Multi-view Representation Learning
by: Yacobi, Amitai, et al.
Published: (2024)

Gradient Free Deep Reinforcement Learning With TabPFN
by: Schiff, David, et al.
Published: (2025)

TempoControl: Temporal Attention Guidance for Text-to-Video Models
by: Schiber, Shira, et al.
Published: (2025)

Unsupervised Acoustic Scene Mapping Based on Acoustic Features and Dimensionality Reduction
by: Cohen, Idan, et al.
Published: (2023)

Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels
by: Grinberg, Yuval, et al.
Published: (2025)

From Segments to Concepts: Interpretable Image Classification via Concept-Guided Segmentation
by: Eisenberg, Ran, et al.
Published: (2025)

Unsupervised Feature Selection Through Group Discovery
by: Lifshitz, Shira, et al.
Published: (2025)

Domain-Generalizable Multiple-Domain Clustering
by: Rozner, Amit, et al.
Published: (2023)

Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching
by: Attia, Amit, et al.
Published: (2024)

More is Less: Inducing Sparsity via Overparameterization
by: Chou, Hung-Hsu, et al.
Published: (2021)

Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation
by: Hill, Adrian, et al.
Published: (2025)

Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
by: Shoham, Ofir Ben
Published: (2026)

On-the-Fly OVD Adaptation with FLAME: Few-shot Localization via Active Marginal-Samples Exploration
by: Refael, Yehonathan, et al.
Published: (2025)

Self-Ablating Transformers: More Interpretability, Less Sparsity
by: Ferrao, Jeremias, et al.
Published: (2025)

An Efficient Training Algorithm for Models with Block-wise Sparsity
by: Zhu, Ding, et al.
Published: (2025)

Mashup Learning: Faster Finetuning by Remixing Past Checkpoints
by: Vaina, Sofia Maria Lo Cicero, et al.
Published: (2026)

HashAttention: Semantic Sparsity for Faster Inference
by: Desai, Aditya, et al.
Published: (2024)

Anomaly Detection with Variance Stabilized Density Estimation
by: Rozner, Amit, et al.
Published: (2023)

Extreme Model Compression with Structured Sparsity at Low Precision
by: Liu, Dan, et al.
Published: (2025)

Multimodal Web Navigation with Instruction-Finetuned Foundation Models
by: Furuta, Hiroki, et al.
Published: (2023)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
by: Lou, Chao, et al.
Published: (2024)

Drop-Muon: Update Less, Converge Faster
by: Gruntkowska, Kaja, et al.
Published: (2025)