Cover Image

Dynamic Low-rank Approximation of Full-Matrix Preconditioner for Training Generalized Linear Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Matveeva, Tatyana, Katrutsa, Aleksandr, Frolov, Evgeny
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Empirical evaluation of the Frank-Wolfe methods for constructing white-box adversarial attacks
by: Korotkova, Kristina, et al.
Published: (2025)

Generative modeling of Sparse Approximate Inverse Preconditioners
by: Li, Mou, et al.
Published: (2024)

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
by: Chen, Xi, et al.
Published: (2024)

Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
by: Chekalina, Viktoriia, et al.
Published: (2025)

Functional multi-armed bandit and the best function identification problems
by: Dorn, Yuriy, et al.
Published: (2025)

Tensor Train Low-rank Approximation (TT-LoRA): Democratizing AI with Accelerated LLMs
by: Anjum, Afia, et al.
Published: (2024)

End-to-End Graph-Sequential Representation Learning for Accurate Recommendations
by: Baikalov, Vladimir, et al.
Published: (2024)

Position-Aware Sequential Attention for Accurate Next Item Recommendations
by: Nabiev, Timur, et al.
Published: (2026)

Tensor Low-rank Approximation of Finite-horizon Value Functions
by: Rozada, Sergio, et al.
Published: (2024)

Matrix Low-Rank Approximation For Policy Gradient Methods
by: Rozada, Sergio, et al.
Published: (2024)

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning
by: Rozada, Sergio, et al.
Published: (2022)

Knowledge Graph Completion with Mixed Geometry Tensor Factorization
by: Yusupov, Viacheslav, et al.
Published: (2025)

On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
by: Ye, Hao, et al.
Published: (2026)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
by: Huang, Jia-Hong, et al.
Published: (2024)

Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler
by: Dremov, Aleksandr, et al.
Published: (2025)

Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds
by: Xu, Conglong, et al.
Published: (2025)

Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
by: Liang, Yingyu, et al.
Published: (2024)

Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner
by: Eschenhagen, Runa, et al.
Published: (2025)

SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression
by: Mozaffari, Mohammad, et al.
Published: (2024)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
by: Zhang, Chengqian, et al.
Published: (2026)

Compute-Optimal Quantization-Aware Training
by: Dremov, Aleksandr, et al.
Published: (2025)

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
by: Li, Junlin, et al.
Published: (2026)

Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations
by: Xu, Alec S., et al.
Published: (2026)

Unveiling the Training Dynamics of ReLU Networks through a Linear Lens
by: Ye, Longqing
Published: (2025)

Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
by: Alimaskina, Ekaterina, et al.
Published: (2026)

NeuraLSP: An Efficient and Rigorous Neural Left Singular Subspace Preconditioner for Conjugate Gradient Methods
by: Benanti, Alexander, et al.
Published: (2026)

Misspecified $Q$-Learning with Sparse Linear Function Approximation: Tight Bounds on Approximation Error
by: Du, Ally Yalei, et al.
Published: (2024)

TRAM: Training Approximate Multiplier Structures for Low-Power AI Accelerators
by: Meng, Chang, et al.
Published: (2026)

Optimizer-Induced Low-Dimensional Drift and Transverse Dynamics in Transformer Training
by: Xu, Yongzhong
Published: (2026)

TLoRA: Tri-Matrix Low-Rank Adaptation of Large Language Models
by: Islam, Tanvir
Published: (2025)

M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
by: Mishra, Mayank, et al.
Published: (2026)

Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data
by: Liu, Junkang, et al.
Published: (2026)

Large-Step Training Dynamics of a Two-Factor Linear Transformer Model
by: Balasubramanian, Krishnakumar
Published: (2026)

Cross-Representation Knowledge Transfer for Improved Sequential Recommendations
by: Gimranov, Artur, et al.
Published: (2026)

Low-rank Optimization Trajectories Modeling for LLM RLVR Acceleration
by: Chen, Zhipeng, et al.
Published: (2026)

Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization
by: Xu, Ziqing, et al.
Published: (2025)

ST-LoRA: Low-rank Adaptation for Spatio-Temporal Forecasting
by: Ruan, Weilin, et al.
Published: (2024)

BELLA: Black box model Explanations by Local Linear Approximations
by: Radulovic, Nedeljko, et al.
Published: (2023)

SubZeroCore: A Submodular Approach with Zero Training for Coreset Selection
by: Moser, Brian B., et al.
Published: (2025)

Matrix Low-Rank Trust Region Policy Optimization
by: Rozada, Sergio, et al.
Published: (2024)