Cover Image

SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rajabi, Sahar, Nonta, Nayeema, Rambhatla, Sirisha
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Randomized Gradient Subspaces for Efficient Large Language Model Training
by: Rajabi, Sahar, et al.
Published: (2025)

SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
by: Hossain, Saad, et al.
Published: (2025)

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering
by: Hossain, Saad, et al.
Published: (2026)

CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training
by: Thede, Lukas, et al.
Published: (2026)

Gradient Multi-Normalization for Stateless and Scalable LLM Training
by: Scetbon, Meyer, et al.
Published: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
by: Miao, Tianhao, et al.
Published: (2026)

Tracking the Feature Dynamics in LLM Training: A Mechanistic Study
by: Xu, Yang, et al.
Published: (2024)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
by: Lang, Yicheng, et al.
Published: (2026)

Deep Q-Learning with Gradient Target Tracking
by: Park, Bum Geun, et al.
Published: (2025)

Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy
by: Li, Tao, et al.
Published: (2024)

Log Probability Tracking of LLM APIs
by: Chauvin, Timothée, et al.
Published: (2025)

Optimized Gradient Tracking for Decentralized Online Learning
by: Sharma, Shivangi Dubey, et al.
Published: (2023)

PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
by: Li, Yanyi, et al.
Published: (2026)

Diffusion Autoencoders are Scalable Image Tokenizers
by: Chen, Yinbo, et al.
Published: (2025)

Memory-Efficient LLM Training with Online Subspace Descent
by: Liang, Kaizhao, et al.
Published: (2024)

SOMP: Scalable Gradient Inversion for Large Language Models via Subspace-Guided Orthogonal Matching Pursuit
by: Li, Yibo, et al.
Published: (2026)

Identifying Policy Gradient Subspaces
by: Schneider, Jan, et al.
Published: (2024)

Robust Decentralized Learning with Local Updates and Gradient Tracking
by: Ghiasvand, Sajjad, et al.
Published: (2024)

A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization
by: Xie, Furan, et al.
Published: (2025)

Accelerated Gradient Tracking over Time-varying Graphs for Decentralized Optimization
by: Li, Huan, et al.
Published: (2021)

SelfEval: Leveraging the discriminative nature of generative models for evaluation
by: Rambhatla, Sai Saketh, et al.
Published: (2023)

EnviroLLM: Resource Tracking and Optimization for Local AI
by: Allen, Troy
Published: (2025)

Muon is Scalable for LLM Training
by: Liu, Jingyuan, et al.
Published: (2025)

Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Tracking
by: Chen, Jun, et al.
Published: (2025)

High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking
by: Armacki, Aleksandar, et al.
Published: (2026)

Enhancing Convergence of Decentralized Gradient Tracking under the KL Property
by: Chen, Xiaokai, et al.
Published: (2024)

Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networks
by: Nguyen, Duong Thuy Anh, et al.
Published: (2024)

Fast and Scalable Semi-Supervised Learning for Multi-View Subspace Clustering
by: Ling, Huaming, et al.
Published: (2024)

BootsTAP: Bootstrapped Training for Tracking-Any-Point
by: Doersch, Carl, et al.
Published: (2024)

FuSeFL: Fully Secure and Scalable Federated Learning
by: Ghinani, Sahar Ghoflsaz, et al.
Published: (2025)

Fast Decentralized Gradient Tracking for Federated Minimax Optimization with Local Updates
by: Li, Chris Junchi
Published: (2024)

Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropout
by: Dezhboro, Amirhossein, et al.
Published: (2026)

$μ$nit Scaling: Simple and Scalable FP8 LLM Training
by: Narayan, Saaketh, et al.
Published: (2025)

SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
by: Refael, Yehonathan, et al.
Published: (2025)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
by: Li, Lujun, et al.
Published: (2025)

Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum
by: Zhou, Yuan, et al.
Published: (2025)

Track-MDP: Reinforcement Learning for Target Tracking with Controlled Sensing
by: Subramaniam, Adarsh M., et al.
Published: (2024)

Tracking the Best Expert Privately
by: Saha, Aadirupa, et al.
Published: (2025)

Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning
by: Zhu, Shijie, et al.
Published: (2025)

Mixture-of-Experts with Gradient Conflict-Driven Subspace Topology Pruning for Emergent Modularity
by: Gan, Yuxing, et al.
Published: (2025)