Image de couverture de livre

Approximated Orthogonal Projection Unit: Stabilizing Regression Network Training Using Natural Gradient

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Shaoqi, Yang, Chunjie, Lou, Siwei
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
par: Lu, Yishun, et autres
Publié: (2025)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
par: Garg, Ishir, et autres
Publié: (2026)

ONG: Orthogonal Natural Gradient Descent
par: Yadav, Yajat, et autres
Publié: (2025)

Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
par: Zhang, Xiyang, et autres
Publié: (2026)

Improving Autoformalization Using Direct Dependency Retrieval
par: Wang, Shaoqi, et autres
Publié: (2025)

Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space
par: Zixian, Wang
Publié: (2026)

Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection
par: Yang, Ziyu, et autres
Publié: (2026)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
par: He, Wei, et autres
Publié: (2025)

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
par: Yang, Jing, et autres
Publié: (2025)

Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
par: Zixian, Wang
Publié: (2026)

Harnessing Orthogonality to Train Low-Rank Neural Networks
par: Coquelin, Daniel, et autres
Publié: (2024)

Gradient Aligned Regression via Pairwise Losses
par: Zhu, Dixian, et autres
Publié: (2024)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
par: Huang, Jia-Hong, et autres
Publié: (2024)

Evaluating and Designing Sparse Autoencoders by Approximating Quasi-Orthogonality
par: Lee, Sewoong, et autres
Publié: (2025)

Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learning
par: Huo, Yingxiao, et autres
Publié: (2026)

Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learning
par: Ma, Guoqing, et autres
Publié: (2025)

GradientStabilizer:Fix the Norm, Not the Gradient
par: Huang, Tianjin, et autres
Publié: (2025)

Gradient-Free Training of Quantized Neural Networks
par: Cohen, Noa, et autres
Publié: (2024)

Gradient Regularized Natural Gradients
par: Dash, Satya Prakash, et autres
Publié: (2026)

GAC: Stabilizing Asynchronous RL Training for LLMs via Gradient Alignment Control
par: Xu, Haofeng, et autres
Publié: (2026)

Pro-KLShampoo: Projected KL-Shampoo with Whitening Recovered by Orthogonalization
par: Sun, Ruotong, et autres
Publié: (2026)

KAIROS: Unified Training for Universal Non-Autoregressive Time Series Forecasting
par: Ding, Kuiye, et autres
Publié: (2025)

Automatic Stability and Recovery for Neural Network Training
par: Or, Barak
Publié: (2026)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
par: Su, DiJia, et autres
Publié: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
par: Miao, Tianhao, et autres
Publié: (2026)

GoQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantization
par: Xiang, Maoyang, et autres
Publié: (2026)

Benign Overfitting for Regression with Trained Two-Layer ReLU Networks
par: Park, Junhyung, et autres
Publié: (2024)

LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration
par: Qiu, Ruiyu, et autres
Publié: (2025)

Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation
par: Che, Fengdi, et autres
Publié: (2024)

Unbiased Gradient Low-Rank Projection
par: Pan, Rui, et autres
Publié: (2025)

Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent
par: Liu, Weihua, et autres
Publié: (2024)

Vertical Symbolic Regression via Deep Policy Gradient
par: Jiang, Nan, et autres
Publié: (2024)

Research and application of Transformer based anomaly detection model: A literature review
par: Ma, Mingrui, et autres
Publié: (2024)

Enhancing Transformer-based models for Long Sequence Time Series Forecasting via Structured Matrix
par: Zhang, Zhicheng, et autres
Publié: (2024)

Orthogonal Subspace Projection for Continual Machine Unlearning via SVD-Based LoRA
par: Rahulamathavan, Yogachandran, et autres
Publié: (2026)

Test-Time Training on Graphs with Large Language Models (LLMs)
par: Zhang, Jiaxin, et autres
Publié: (2024)

Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
par: Xiao, Mingqing, et autres
Publié: (2024)

Gradient-Congruity Guided Federated Sparse Training
par: Tian, Chris Xing, et autres
Publié: (2024)

Matrix Low-Rank Approximation For Policy Gradient Methods
par: Rozada, Sergio, et autres
Publié: (2024)

Enhancing Stability for Large Language Models Training in Constrained Bandwidth Networks
par: Dai, Yun, et autres
Publié: (2024)