Cover Image

Low Rank Gradients and Where to Find Them

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sonthalia, Rishi, Murray, Michael, Montúfar, Guido
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
by: Li, Jiping, et al.
Published: (2025)

Fantastic Pretraining Optimizers and Where to Find Them
by: Wen, Kaiyue, et al.
Published: (2025)

Fantastic Bugs and Where to Find Them in AI Benchmarks
by: Truong, Sang, et al.
Published: (2025)

Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis
by: Datta, Shrestha, et al.
Published: (2026)

Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process
by: Zhang, Zhenyu, et al.
Published: (2025)

Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them
by: Bui, Anh, et al.
Published: (2025)

What Cohort INRs Encode and Where to Freeze Them
by: Sideri-Lampretsa, Vasiliki, et al.
Published: (2026)

Flora: Low-Rank Adapters Are Secretly Gradient Compressors
by: Hao, Yongchang, et al.
Published: (2024)

Matrix Low-Rank Approximation For Policy Gradient Methods
by: Rozada, Sergio, et al.
Published: (2024)

Calibrated Language Models and How to Find Them with Label Smoothing
by: Huang, Jerry, et al.
Published: (2025)

Fantastic Multi-Task Gradient Updates and How to Find Them In a Cone
by: Hassanpour, Negar, et al.
Published: (2025)

Unbiased Gradient Low-Rank Projection
by: Pan, Rui, et al.
Published: (2025)

Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)
by: Prinster, Drew, et al.
Published: (2024)

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets
by: Devine, Peter
Published: (2024)

Do Deep Neural Network Solutions Form a Star Domain?
by: Sonthalia, Ankit, et al.
Published: (2024)

On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective
by: Xie, Zeke, et al.
Published: (2020)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
by: Su, DiJia, et al.
Published: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
by: Miao, Tianhao, et al.
Published: (2026)

GoRA: Gradient-driven Adaptive Low Rank Adaptation
by: He, Haonan, et al.
Published: (2025)

Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization
by: Xu, Ziqing, et al.
Published: (2025)

Low Tensor-Rank Adaptation of Kolmogorov--Arnold Networks
by: Gao, Yihang, et al.
Published: (2025)

Generalization for Least Squares Regression With Simple Spiked Covariances
by: Li, Jiping, et al.
Published: (2024)

Least Squares Regression Can Exhibit Under-Parameterized Double Descent
by: Li, Xinyue, et al.
Published: (2023)

Low-Rank Compression of Language Models via Differentiable Rank Selection
by: Sundrani, Sidhant, et al.
Published: (2025)

Towards Symmetric Low-Rank Adapters
by: Panoutsos, Tales, et al.
Published: (2025)

The Primacy of Magnitude in Low-Rank Adaptation
by: Zhang, Zicheng, et al.
Published: (2025)

Mixture-of-Subspaces in Low-Rank Adaptation
by: Wu, Taiqiang, et al.
Published: (2024)

Continual Gradient Low-Rank Projection Fine-Tuning for LLMs
by: Wang, Chenxu, et al.
Published: (2025)

Global Low-Rank, Local Full-Rank: The Holographic Encoding of Learned Algorithms
by: Xu, Yongzhong
Published: (2026)

Probe-Free Low-Rank Activation Intervention
by: Jiang, Chonghe, et al.
Published: (2025)

Low-Rank MDPs with Continuous Action Spaces
by: Bennett, Andrew, et al.
Published: (2023)

Low-Perplexity LLM-Generated Sequences and Where To Find Them
by: Wuhrmann, Arthur, et al.
Published: (2025)

Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region
by: Liang, Shuang, et al.
Published: (2025)

Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
by: Zhang, Wenhao, et al.
Published: (2026)

Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
by: Patankar, Dhruv, et al.
Published: (2026)

Low-Rank Tensor Decompositions for the Theory of Neural Networks
by: Borsoi, Ricardo, et al.
Published: (2025)

Few-Shot Concept Unlearning with Low Rank Adaptation
by: Shreyas, Udaya, et al.
Published: (2025)

Hierarchical Sparse Plus Low Rank Compression of LLM
by: Kumar, Pawan, et al.
Published: (2025)

Aggregating Low Rank Adapters in Federated Fine-tuning
by: Trautmann, Evelyn, et al.
Published: (2025)

PLAN: Proactive Low-Rank Allocation for Continual Learning
by: Wang, Xiequn, et al.
Published: (2025)