Copertina

First-ish Order Methods: Hessian-aware Scalings of Gradient Descent

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Smee, Oscar, Roosta, Fred, Wright, Stephen J.
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Optimization and Control Machine Learning 49
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints
di: Smee, Oscar, et al.
Pubblicazione: (2024)

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms
di: Lu, Eric
Pubblicazione: (2025)

Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
di: Chiang, John
Pubblicazione: (2022)

On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation
di: Kwon, Jeongyeol, et al.
Pubblicazione: (2023)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
di: Gupta, Devansh, et al.
Pubblicazione: (2025)

Complexity Guarantees for Nonconvex Newton-MR Under Inexact Hessian Information
di: Lim, Alexander, et al.
Pubblicazione: (2023)

On Penalty-based Bilevel Gradient Descent Method
di: Shen, Han, et al.
Pubblicazione: (2023)

Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law
di: Kunstner, Frederik, et al.
Pubblicazione: (2025)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)

Training Autoencoders Using Stochastic Hessian-Free Optimization with LSMR
di: Emirahmetoglu, Ibrahim, et al.
Pubblicazione: (2025)

Corner Gradient Descent
di: Yarotsky, Dmitry
Pubblicazione: (2025)

Gradient-Normalized Smoothness for Optimization with Approximate Hessians
di: Semenov, Andrei, et al.
Pubblicazione: (2025)

More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems
di: Partohaghighi, Mohammad, et al.
Pubblicazione: (2025)

Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems
di: Partohaghighi, Mohammad, et al.
Pubblicazione: (2025)

Convergence Rate for the Last Iterate of Stochastic Gradient Descent Schemes
di: Hudiani, Marcel
Pubblicazione: (2025)

Adaptive Conditional Gradient Descent
di: Khademi, Abbas, et al.
Pubblicazione: (2025)

$k$-SVD with Gradient Descent
di: Jedra, Yassir, et al.
Pubblicazione: (2025)

DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method
di: Khaled, Ahmed, et al.
Pubblicazione: (2023)

Anytime Acceleration of Gradient Descent
di: Zhang, Zihan, et al.
Pubblicazione: (2024)

Gradient Methods with Online Scaling
di: Gao, Wenzhi, et al.
Pubblicazione: (2024)

Efficiently Training Deep-Learning Parametric Policies using Lagrangian Duality
di: Rosemberg, Andrew, et al.
Pubblicazione: (2024)

A Short and General Duality Proof for Wasserstein Distributionally Robust Optimization
di: Zhang, Luhao, et al.
Pubblicazione: (2022)

Stochastic Gradient Descent with Strategic Querying
di: Jiang, Nanfei, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent with Adaptive Data
di: Che, Ethan, et al.
Pubblicazione: (2024)

Unraveling the Gradient Descent Dynamics of Transformers
di: Song, Bingqing, et al.
Pubblicazione: (2024)

Zero-Order Optimization for Gaussian Process-based Model Predictive Control
di: Lahr, Amon, et al.
Pubblicazione: (2022)

Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points
di: Yamamoto, Naoya, et al.
Pubblicazione: (2025)

Hellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionals
di: Mielke, Alexander, et al.
Pubblicazione: (2025)

The Method of Infinite Descent
di: Batley, Reza T., et al.
Pubblicazione: (2025)

Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods
di: O'Leary-Roseberry, Thomas, et al.
Pubblicazione: (2024)

Accelerated Stochastic ExtraGradient: Mixing Hessian and Gradient Similarity to Reduce Communication in Distributed and Federated Learning
di: Bylinkin, Dmitry, et al.
Pubblicazione: (2024)

Learning Provably Improves the Convergence of Gradient Descent
di: Song, Qingyu, et al.
Pubblicazione: (2025)

Enhancing Fractional Gradient Descent with Learned Optimizers
di: Sobotka, Jan, et al.
Pubblicazione: (2025)

Mirror and Preconditioned Gradient Descent in Wasserstein Space
di: Bonet, Clément, et al.
Pubblicazione: (2024)

Derivatives of Stochastic Gradient Descent in parametric optimization
di: Iutzeler, Franck, et al.
Pubblicazione: (2024)

Convergence of Alternating Gradient Descent for Matrix Factorization
di: Ward, Rachel, et al.
Pubblicazione: (2023)

A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models
di: Xu, Ziqing, et al.
Pubblicazione: (2025)

Robustness of Iteratively Pre-Conditioned Gradient-Descent Method: The Case of Distributed Linear Regression Problem
di: Chakrabarti, Kushal, et al.
Pubblicazione: (2021)

Iterative Pre-Conditioning for Expediting the Gradient-Descent Method: The Distributed Linear Least-Squares Problem
di: Chakrabarti, Kushal, et al.
Pubblicazione: (2020)

Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reduction
di: Feng, Jie, et al.
Pubblicazione: (2024)