Copertina

AdaFisher: Adaptive Second Order Optimization via Fisher Information

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Gomes, Damien Martins, Zhang, Yanlei, Belilovsky, Eugene, Wolf, Guy, Hosseini, Mahdi S.
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis
di: Gomes, Damien Martins
Pubblicazione: (2025)

A Split-Client Approach to Second-Order Optimization
di: Chayti, El Mahdi, et al.
Pubblicazione: (2025)

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow
di: Ren, Yinuo, et al.
Pubblicazione: (2023)

RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
di: Chayti, El Mahdi
Pubblicazione: (2026)

PPO in the Fisher-Rao geometry
di: Lascu, Razvan-Andrei, et al.
Pubblicazione: (2025)

TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
di: Li, Xiang, et al.
Pubblicazione: (2022)

Inclusive KL Minimization: A Wasserstein-Fisher-Rao Gradient Flow Perspective
di: Zhu, Jia-Jie
Pubblicazione: (2024)

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
di: Zhang, Minxin, et al.
Pubblicazione: (2025)

On Adaptivity in Zeroth-Order Optimization
di: Dbouk, Hassan, et al.
Pubblicazione: (2026)

First-Order Methods for Linearly Constrained Bilevel Optimization
di: Kornowski, Guy, et al.
Pubblicazione: (2024)

An Algorithm with Optimal Dimension-Dependence for Zero-Order Nonsmooth Nonconvex Stochastic Optimization
di: Kornowski, Guy, et al.
Pubblicazione: (2023)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2024)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
di: Ostroukhov, Petr, et al.
Pubblicazione: (2024)

On propagation of chaos for the Fisher-Rao gradient flow in entropic mean-field optimization
di: Lazić, Petra, et al.
Pubblicazione: (2026)

Better LMO-based Momentum Methods with Second-Order Information
di: Khirirat, Sarit, et al.
Pubblicazione: (2025)

Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity
di: Zhou, Qihao, et al.
Pubblicazione: (2024)

Optimization with Access to Auxiliary Information
di: Chayti, El Mahdi, et al.
Pubblicazione: (2022)

A Fisher-Rao gradient flow for entropic mean-field min-max games
di: Lascu, Razvan-Andrei, et al.
Pubblicazione: (2024)

Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe
di: Chayti, El Mahdi
Pubblicazione: (2026)

AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
di: Bojovic, Matia, et al.
Pubblicazione: (2026)

Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Mixing Time of the Proximal Sampler in Relative Fisher Information via Strong Data Processing Inequality
di: Wibisono, Andre
Pubblicazione: (2025)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
di: Lin, Wu, et al.
Pubblicazione: (2024)

GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms
di: Eleh, Chinedu, et al.
Pubblicazione: (2024)

AdaGrad under Anisotropic Smoothness
di: Liu, Yuxing, et al.
Pubblicazione: (2024)

AdaSwitch: An Adaptive Switching Meta-Algorithm for Learning-Augmented Bounded-Influence Problems
di: Chen, Xi, et al.
Pubblicazione: (2025)

Stochastic Online Fisher Markets: Static Pricing Limits and Adaptive Enhancements
di: Jalota, Devansh, et al.
Pubblicazione: (2022)

Simultaneous Learning and Optimization via Misspecified Saddle Point Problems
di: Ahmadi, Mohammad Mahdi, et al.
Pubblicazione: (2025)

A New First-Order Meta-Learning Algorithm with Convergence Guarantees
di: Chayti, El Mahdi, et al.
Pubblicazione: (2024)

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
di: Liu, Zijian
Pubblicazione: (2026)

A Second-Order Majorant Algorithm for Nonnegative Matrix Factorization
di: Pham, Mai-Quyen, et al.
Pubblicazione: (2023)

Second-Order Min-Max Optimization with Lazy Hessians
di: Chen, Lesi, et al.
Pubblicazione: (2024)

Revisiting Convergence of AdaGrad with Relaxed Assumptions
di: Hong, Yusu, et al.
Pubblicazione: (2024)

Don't Be So Positive: Negative Step Sizes in Second-Order Methods
di: Shea, Betty, et al.
Pubblicazione: (2024)

Stochastic Difference-of-Convex Optimization with Momentum
di: Chayti, El Mahdi, et al.
Pubblicazione: (2025)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Universal Online Convex Optimization Meets Second-order Bounds
di: Zhang, Lijun, et al.
Pubblicazione: (2021)

Scalable Second-order Riemannian Optimization for $K$-means Clustering
di: Xu, Peng, et al.
Pubblicazione: (2025)

On the Complexity of Finding Small Subgradients in Nonsmooth Optimization
di: Kornowski, Guy, et al.
Pubblicazione: (2022)

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms
di: Lu, Eric
Pubblicazione: (2025)