Cover Image

FineGates: LLMs Finetuning with Compression using Stochastic Gates

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Svirsky, Jonathan, Refael, Yehonathan, Lindenbaum, Ofir
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Train Less, Infer Faster: Efficient Model Finetuning and Compression via Structured Sparsity
by: Svirsky, Jonathan, et al.
Published: (2026)

AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning
by: Refael, Yehonathan, et al.
Published: (2024)

Interpretable Deep Clustering for Tabular Data
by: Svirsky, Jonathan, et al.
Published: (2023)

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
by: Arbel, Iftach, et al.
Published: (2024)

Unveiling Multiple Descents in Unsupervised Autoencoders
by: Rahimi, Kobi, et al.
Published: (2024)

LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
by: Refael, Yehonathan, et al.
Published: (2025)

SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
by: Refael, Yehonathan, et al.
Published: (2025)

Self Supervised Correlation-based Permutations for Multi-View Clustering
by: Eisenberg, Ran, et al.
Published: (2024)

Sparse Binarization for Fast Keyword Spotting
by: Svirsky, Jonathan, et al.
Published: (2024)

Uncovering a Winning Lottery Ticket with Continuously Relaxed Bernoulli Gates
by: Tsayag, Itamar, et al.
Published: (2026)

Contextual Feature Selection with Conditional Stochastic Gates
by: Sristi, Ram Dyuthi, et al.
Published: (2023)

No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks
by: Refael, Yehonatan, et al.
Published: (2025)

Mathematical Framework for Online Social Media Auditing
by: Huleihel, Wasim, et al.
Published: (2022)

Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization
by: Refael, Yehonathan, et al.
Published: (2022)

Provable Speech Attributes Conversion via Latent Independence
by: Svirsky, Jonathan, et al.
Published: (2025)

Learning Permutation from Structure Without Supervision
by: Eisenberg, Ran, et al.
Published: (2026)

Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings
by: Naor, Erel, et al.
Published: (2025)

Conditional Deep Canonical Time Warping
by: Steinberg, Afek, et al.
Published: (2024)

Spectral Self-supervised Feature Selection
by: Segal, Daniel, et al.
Published: (2024)

Generalizable and Robust Spectral Method for Multi-view Representation Learning
by: Yacobi, Amitai, et al.
Published: (2024)

Gradient Free Deep Reinforcement Learning With TabPFN
by: Schiff, David, et al.
Published: (2025)

Unsupervised Acoustic Scene Mapping Based on Acoustic Features and Dimensionality Reduction
by: Cohen, Idan, et al.
Published: (2023)

Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels
by: Grinberg, Yuval, et al.
Published: (2025)

TempoControl: Temporal Attention Guidance for Text-to-Video Models
by: Schiber, Shira, et al.
Published: (2025)

SLIP: Securing LLMs IP Using Weights Decomposition
by: Refael, Yehonathan, et al.
Published: (2024)

Unsupervised Feature Selection Through Group Discovery
by: Lifshitz, Shira, et al.
Published: (2025)

Domain-Generalizable Multiple-Domain Clustering
by: Rozner, Amit, et al.
Published: (2023)

From Segments to Concepts: Interpretable Image Classification via Concept-Guided Segmentation
by: Eisenberg, Ran, et al.
Published: (2025)

Enhancing User Experience in On-Device Machine Learning with Gated Compression Layers
by: Li, Haiguang, et al.
Published: (2024)

GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMs
by: Zhang, Xuanqi, et al.
Published: (2026)

VeriGate: Verifier-Gated Step-Level Supervision for GRPO
by: Agrawal, Aakriti, et al.
Published: (2026)

Anomaly Detection with Variance Stabilized Density Estimation
by: Rozner, Amit, et al.
Published: (2023)

GatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
by: Liu, Jiaxu, et al.
Published: (2025)

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

SigGate: Enhancing Recurrent Neural Networks with Signature-Based Gating Mechanisms
by: Genet, Rémi, et al.
Published: (2025)

In-context KV-Cache Eviction for LLMs via Attention-Gate
by: Zeng, Zihao, et al.
Published: (2024)

GateRA: Token-Aware Modulation for Parameter-Efficient Fine-Tuning
by: Ou, Jie, et al.
Published: (2025)

ArcGate: Adaptive Arctangent Gated Activation
by: Bhattacharya, Avik, et al.
Published: (2026)

Uncertainty Estimation using Variance-Gated Distributions
by: Gillis, H. Martin, et al.
Published: (2025)

CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planning
by: Wang, Jun, et al.
Published: (2025)