Cover Image

Gradient Regularized Natural Gradients

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dash, Satya Prakash, Abdi, Hossein, Pan, Wei, Kaski, Samuel, Sun, Mingfei
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learning
by: Huo, Yingxiao, et al.
Published: (2026)

Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering
by: Abdi, Hossein, et al.
Published: (2025)

Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagation
by: Sun, Mingfei
Published: (2026)

LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
by: Abdi, Hossein, et al.
Published: (2024)

ONG: Orthogonal Natural Gradient Descent
by: Yadav, Yajat, et al.
Published: (2025)

Wasserstein Gradient Flows for Scalable and Regularized Barycenter Computation
by: Montesuma, Eduardo Fernandes, et al.
Published: (2025)

In-Context Black-Box Optimization with Unreliable Feedback
by: Blumer, Nicolas Samuel, et al.
Published: (2026)

Gradient Residual Connections
by: Pan, Yangchen, et al.
Published: (2026)

Transformer Normalisation Layers and the Independence of Semantic Subspaces
by: Menary, Stephen, et al.
Published: (2024)

Policy Gradient Methods in the Presence of Symmetries and State Abstractions
by: Panangaden, Prakash, et al.
Published: (2023)

Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization
by: Liu, Kezhao, et al.
Published: (2025)

Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning
by: Yang, Tong, et al.
Published: (2023)

GradientStabilizer:Fix the Norm, Not the Gradient
by: Huang, Tianjin, et al.
Published: (2025)

AFBS:Buffer Gradient Selection in Semi-asynchronous Federated Learning
by: Lu, Chaoyi, et al.
Published: (2025)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
by: Garg, Ishir, et al.
Published: (2026)

Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
by: Kumar, Navish, et al.
Published: (2025)

Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
by: Cao, Jiarui, et al.
Published: (2026)

Measures of Variability for Risk-averse Policy Gradient
by: Luo, Yudong, et al.
Published: (2025)

Fast Explanations via Policy Gradient-Optimized Explainer
by: Pan, Deng, et al.
Published: (2024)

Elastic Multi-Gradient Descent for Parallel Continual Learning
by: Lyu, Fan, et al.
Published: (2024)

On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
by: Zhang, Yifan, et al.
Published: (2025)

Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
by: Sheen, Heejune, et al.
Published: (2024)

The Reciprocity Gradient
by: Lin, Yue, et al.
Published: (2026)

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
by: Liang, Chen, et al.
Published: (2026)

Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent
by: Liu, Weihua, et al.
Published: (2024)

Gradient Routing: Masking Gradients to Localize Computation in Neural Networks
by: Cloud, Alex, et al.
Published: (2024)

When Will Gradient Regularization Be Harmful?
by: Zhao, Yang, et al.
Published: (2024)

On Spectral Properties of Gradient-based Explanation Methods
by: Mehrpanah, Amir, et al.
Published: (2025)

Approximated Orthogonal Projection Unit: Stabilizing Regression Network Training Using Natural Gradient
by: Wang, Shaoqi, et al.
Published: (2024)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
by: Lu, Yishun, et al.
Published: (2025)

Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation
by: Dai, Juntao, et al.
Published: (2024)

On Task Vectors and Gradients
by: Zhou, Luca, et al.
Published: (2025)

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
by: Chen, Yihong, et al.
Published: (2026)

More Than Irrational: Modeling Belief-Biased Agents
by: Zhu, Yifan, et al.
Published: (2025)

In-Context Multi-Objective Optimization
by: Zhang, Xinyu, et al.
Published: (2025)

The Devil Is in Gradient Entanglement: Energy-Aware Gradient Coordinator for Robust Generalized Category Discovery
by: Zheng, Haiyang, et al.
Published: (2026)

Beyond Gradient Averaging in Parallel Optimization: Improved Robustness through Gradient Agreement Filtering
by: Chaubard, Francois, et al.
Published: (2024)

Unbiased Gradient Low-Rank Projection
by: Pan, Rui, et al.
Published: (2025)

Tackling the Non-IID Issue in Heterogeneous Federated Learning by Gradient Harmonization
by: Zhang, Xinyu, et al.
Published: (2023)

Does This Gradient Spark Joy?
by: Osband, Ian
Published: (2026)