Cover Image

Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Gomes, Damien Martins
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AdaFisher: Adaptive Second Order Optimization via Fisher Information
by: Gomes, Damien Martins, et al.
Published: (2024)

A Split-Client Approach to Second-Order Optimization
by: Chayti, El Mahdi, et al.
Published: (2025)

Better LMO-based Momentum Methods with Second-Order Information
by: Khirirat, Sarit, et al.
Published: (2025)

Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity
by: Zhou, Qihao, et al.
Published: (2024)

RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
by: Chayti, El Mahdi
Published: (2026)

Optimization Insights into Deep Diagonal Linear Networks
by: Labarrière, Hippolyte, et al.
Published: (2024)

Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning
by: Tran, Hoang, et al.
Published: (2024)

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow
by: Ren, Yinuo, et al.
Published: (2023)

A Second-Order Majorant Algorithm for Nonnegative Matrix Factorization
by: Pham, Mai-Quyen, et al.
Published: (2023)

Deep Reinforcement Learning for Dynamic Order Picking in Warehouse Operations
by: Mahmoudinazlou, Sasan, et al.
Published: (2024)

Don't Be So Positive: Negative Step Sizes in Second-Order Methods
by: Shea, Betty, et al.
Published: (2024)

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms
by: Lu, Eric
Published: (2025)

PPO in the Fisher-Rao geometry
by: Lascu, Razvan-Andrei, et al.
Published: (2025)

On Adaptivity in Zeroth-Order Optimization
by: Dbouk, Hassan, et al.
Published: (2026)

Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization
by: Sahinoglu, Emre, et al.
Published: (2024)

Towards Robust Learning to Optimize with Theoretical Guarantees
by: Song, Qingyu, et al.
Published: (2025)

Policy Gradient with Second Order Momentum
by: Sun, Tianyu
Published: (2025)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
by: Lin, Wu, et al.
Published: (2024)

Learning-to-Optimize with PAC-Bayesian Guarantees: Theoretical Considerations and Practical Implementation
by: Sucker, Michael, et al.
Published: (2024)

Deep Reinforcement Learning: A Convex Optimization Approach
by: Gattami, Ather
Published: (2024)

Learning Constrained Optimization with Deep Augmented Lagrangian Methods
by: Kotary, James, et al.
Published: (2024)

Second-Order Min-Max Optimization with Lazy Hessians
by: Chen, Lesi, et al.
Published: (2024)

Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learning
by: Hu, Yifan, et al.
Published: (2020)

Scalable Second-order Riemannian Optimization for $K$-means Clustering
by: Xu, Peng, et al.
Published: (2025)

Universal Online Convex Optimization Meets Second-order Bounds
by: Zhang, Lijun, et al.
Published: (2021)

Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization
by: Jiang, Ruichen, et al.
Published: (2024)

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
by: Jiang, Ruichen, et al.
Published: (2024)

Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance
by: Chen, Lisha, et al.
Published: (2025)

Solving Convex-Concave Problems with $\tilde{\mathcal{O}}(ε^{-4/7})$ Second-Order Oracle Complexity
by: Chen, Lesi, et al.
Published: (2025)

Fully First-Order Algorithms for Online Bilevel Optimization
by: Jia, Tingkai, et al.
Published: (2026)

On the Complexity of First-Order Methods in Stochastic Bilevel Optimization
by: Kwon, Jeongyeol, et al.
Published: (2024)

First-Order Methods for Linearly Constrained Bilevel Optimization
by: Kornowski, Guy, et al.
Published: (2024)

A Study of Condition Numbers for First-Order Optimization
by: Guille-Escuret, Charles, et al.
Published: (2020)

EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
by: Yau, Chung-Yiu, et al.
Published: (2026)

Bridging Theory and Practice: A Stochastic Learning-Optimization Model for Resilient Automotive Supply Chains
by: Shahnawaz, Muhammad, et al.
Published: (2025)

Towards Fully Parameter-Free Stochastic Optimization: Grid Search with Self-Bounding Analysis
by: Zhao, Yuheng, et al.
Published: (2026)

Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks
by: Qin, Zhen, et al.
Published: (2023)

Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
by: Chen, Ziyi, et al.
Published: (2025)

Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization
by: Li, Chris Junchi
Published: (2024)

Adaptive Moment Estimation Optimization Algorithm Using Projection Gradient for Deep Learning
by: Li, Yongqi, et al.
Published: (2025)