Cover Image

Randomized Gradient Subspaces for Efficient Large Language Model Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rajabi, Sahar, Nonta, Nayeema, Vajpayee, Samanvay, Rambhatla, Sirisha
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training
by: Rajabi, Sahar, et al.
Published: (2025)

SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
by: Hossain, Saad, et al.
Published: (2025)

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering
by: Hossain, Saad, et al.
Published: (2026)

A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
by: Chen, Yiming, et al.
Published: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
by: Miao, Tianhao, et al.
Published: (2026)

PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
by: Li, Yanyi, et al.
Published: (2026)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
by: Lang, Yicheng, et al.
Published: (2026)

Augmenting the FedProx Algorithm by Minimizing Convergence
by: Sarkar, Anomitra, et al.
Published: (2024)

SOMP: Scalable Gradient Inversion for Large Language Models via Subspace-Guided Orthogonal Matching Pursuit
by: Li, Yibo, et al.
Published: (2026)

Exploring Gradient Subspaces: Addressing and Overcoming LoRA's Limitations in Federated Fine-Tuning of Large Language Models
by: Mahla, Navyansh, et al.
Published: (2024)

Efficient Resource-Constrained Training of Transformers via Subspace Optimization
by: Nguyen, Le-Trung, et al.
Published: (2025)

OASIS: Online Activation Subspace Learning for Memory-Efficient Training
by: Choudhary, Sakshi, et al.
Published: (2026)

Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy
by: Li, Tao, et al.
Published: (2024)

Subspace Optimization for Large Language Models with Convergence Guarantees
by: He, Yutong, et al.
Published: (2024)

Deep Hierarchical Learning with Nested Subspace Networks for Large Language Models
by: Rauba, Paulius, et al.
Published: (2025)

Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
by: Zhou, Wenjie, et al.
Published: (2026)

Training Large Language Models to Reason via EM Policy Gradient
by: Xu, Tianbing
Published: (2025)

Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
by: Yuan, Aojie, et al.
Published: (2026)

Differentially Private Subspace Fine-Tuning for Large Language Models
by: Zheng, Lele, et al.
Published: (2026)

ROSA: Random Subspace Adaptation for Efficient Fine-Tuning
by: Hameed, Marawan Gamal Abdel, et al.
Published: (2024)

TRAVEL: Training-Free Retrieval and Alignment for Vision-and-Language Navigation
by: Rajabi, Navid, et al.
Published: (2025)

Identifying Policy Gradient Subspaces
by: Schneider, Jan, et al.
Published: (2024)

PLUMAGE: Probabilistic Low rank Unbiased Min Variance Gradient Estimator for Efficient Large Model Training
by: Haroush, Matan, et al.
Published: (2025)

Randomized Subspace Nesterov Accelerated Gradient
by: Omiya, Gaku, et al.
Published: (2026)

Synthetic Text Generation for Training Large Language Models via Gradient Matching
by: Nguyen, Dang, et al.
Published: (2025)

AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
by: Li, Zhiyuan, et al.
Published: (2026)

CELLM: An Efficient Communication in Large Language Models Training for Federated Learning
by: Vavekanand, Raja, et al.
Published: (2024)

Memory-Efficient Differentially Private Training with Gradient Random Projection
by: Mulrooney, Alex, et al.
Published: (2025)

Memory-Efficient LLM Training with Online Subspace Descent
by: Liang, Kaizhao, et al.
Published: (2024)

SelfEval: Leveraging the discriminative nature of generative models for evaluation
by: Rambhatla, Sai Saketh, et al.
Published: (2023)

LangDA: Building Context-Awareness via Language for Domain Adaptive Semantic Segmentation
by: Liu, Chang, et al.
Published: (2025)

Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Models
by: Wang, Liangyu, et al.
Published: (2025)

Gradient-Free Training of Recurrent Neural Networks using Random Perturbations
by: Fernandez, Jesus Garcia, et al.
Published: (2024)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
by: Abbes, Istabrak, et al.
Published: (2025)

Attacking Large Language Models with Projected Gradient Descent
by: Geisler, Simon, et al.
Published: (2024)

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
by: Yang, Jing, et al.
Published: (2025)

NeuraLSP: An Efficient and Rigorous Neural Left Singular Subspace Preconditioner for Conjugate Gradient Methods
by: Benanti, Alexander, et al.
Published: (2026)

Knowledge Vector Weakening: Efficient Training-free Unlearning for Large Vision-Language Models
by: Kim, Yejin, et al.
Published: (2026)

ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models
by: Li, Xu, et al.
Published: (2026)

SubspaceAD: Training-Free Few-Shot Anomaly Detection via Subspace Modeling
by: Lendering, Camile, et al.
Published: (2026)