Copertina

Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kong, Deyi, Chen, Zaiwei, Zhang, Shuzhong, Mou, Shancong
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
di: Chu, Ya-Chi, et al.
Pubblicazione: (2025)

ADMM Algorithms for Residual Network Training: Convergence Analysis and Parallel Implementation
di: Xu, Jintao, et al.
Pubblicazione: (2023)

Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework
di: Nanda, Phalguni, et al.
Pubblicazione: (2026)

A Minimal-Assumption Analysis of Q-Learning with Time-Varying Policies
di: Nanda, Phalguni, et al.
Pubblicazione: (2025)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
di: Kong, Boao, et al.
Pubblicazione: (2026)

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators
di: Li, Tianyou, et al.
Pubblicazione: (2023)

Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
di: Dong, Youran, et al.
Pubblicazione: (2025)

Achieving $ε^{-2}$ Sample Complexity for Single-Loop Actor-Critic under Minimal Assumptions
di: Hamza, Ishaq, et al.
Pubblicazione: (2026)

Convergence Properties of Natural Gradient Descent for Minimizing KL Divergence
di: Datar, Adwait, et al.
Pubblicazione: (2025)

A Non-Asymptotic Theory of Seminorm Lyapunov Stability: From Deterministic to Stochastic Iterative Algorithms
di: Chen, Zaiwei, et al.
Pubblicazione: (2025)

On the Convergence of Policy in Unregularized Policy Mirror Descent
di: Lin, Dachao, et al.
Pubblicazione: (2022)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
di: Xu, Xianliang, et al.
Pubblicazione: (2024)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
di: Cayci, Semih, et al.
Pubblicazione: (2024)

Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise
di: Chen, Zaiwei, et al.
Pubblicazione: (2023)

Convergence of Spectral Descent for Non-smooth Optimization
di: Yang, Yixuan, et al.
Pubblicazione: (2026)

Learning Provably Improves the Convergence of Gradient Descent
di: Song, Qingyu, et al.
Pubblicazione: (2025)

Convergence of Alternating Gradient Descent for Matrix Factorization
di: Ward, Rachel, et al.
Pubblicazione: (2023)

Convergence Analysis of the Wasserstein Proximal Algorithm beyond Geodesic Convexity
di: Zhu, Shuailong, et al.
Pubblicazione: (2025)

Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
di: Akhtiamov, Danil, et al.
Pubblicazione: (2026)

On the Convergence of Policy Mirror Descent with Temporal Difference Evaluation
di: Liu, Jiacai, et al.
Pubblicazione: (2025)

Open Problem: Anytime Convergence Rate of Gradient Descent
di: Kornowski, Guy, et al.
Pubblicazione: (2024)

On the Convergence of Gradient Descent on Learning Transformers with Residual Connections
di: Qin, Zhen, et al.
Pubblicazione: (2025)

A New Convergence Analysis of Plug-and-Play Proximal Gradient Descent Under Prior Mismatch
di: Xu, Guixian, et al.
Pubblicazione: (2026)

Exponential Convergence of (Stochastic) Gradient Descent for Separable Logistic Regression
di: Kale, Sacchit, et al.
Pubblicazione: (2026)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
di: Vaswani, Sharan, et al.
Pubblicazione: (2026)

Convergence of Policy Mirror Descent Beyond Compatible Function Approximation
di: Sherman, Uri, et al.
Pubblicazione: (2025)

Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation
di: Mishkin, Aaron, et al.
Pubblicazione: (2024)

Convergence of Gradient Descent with Small Initialization for Unregularized Matrix Completion
di: Ma, Jianhao, et al.
Pubblicazione: (2024)

Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
di: Jung, Hyunji, et al.
Pubblicazione: (2025)

Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferential
di: Zheng, Yuping, et al.
Pubblicazione: (2025)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
di: Oowada, Kanata, et al.
Pubblicazione: (2025)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
di: Li, Xiang, et al.
Pubblicazione: (2024)

Convergence Rates for Gradient Descent on the Edge of Stability in Overparametrised Least Squares
di: MacDonald, Lachlan Ewen, et al.
Pubblicazione: (2025)

Gauss-Newton Natural Gradient Descent for Shape Learning
di: King, James, et al.
Pubblicazione: (2026)

A Short and Unified Convergence Analysis of the SAG, SAGA, and IAG Algorithms
di: Zhu, Feng, et al.
Pubblicazione: (2026)

Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
di: Hao, Jie, et al.
Pubblicazione: (2024)

Convergence Analysis of the PAGE Stochastic Algorithm for Weakly Convex Finite-Sum Optimization
di: Condat, Laurent, et al.
Pubblicazione: (2025)

Convergence Analysis of Fractional Gradient Descent
di: Aggarwal, Ashwani
Pubblicazione: (2023)

Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks
di: Beneventano, Pierfrancesco, et al.
Pubblicazione: (2025)

Linearly Convergent Algorithms for Nonsmooth Problems with Unknown Smooth Pieces
di: Zhang, Zhe, et al.
Pubblicazione: (2025)