Copertina

MADA: Meta-Adaptive Optimizers through hyper-gradient Descent

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Ozkara, Kaan, Karakus, Can, Raman, Parameswaran, Hong, Mingyi, Sabach, Shoham, Kveton, Branislav, Cevher, Volkan
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Comparing Few to Rank Many: Active Human Preference Learning using Randomized Frank-Wolfe
di: Thekumparampil, Kiran Koshy, et al.
Pubblicazione: (2024)

MuonBP: Faster Muon via Block-Periodic Orthogonalization
di: Khaled, Ahmed, et al.
Pubblicazione: (2025)

Stable Nonconvex-Nonconcave Training via Linear Interpolation
di: Pethick, Thomas, et al.
Pubblicazione: (2023)

Dynamic FISTA for Convex Composite Bi-Level Optimization
di: Merchav, Roey, et al.
Pubblicazione: (2024)

Unraveling the Gradient Descent Dynamics of Transformers
di: Song, Bingqing, et al.
Pubblicazione: (2024)

Efficient Continual Finite-Sum Minimization
di: Mavrothalassitis, Ioannis, et al.
Pubblicazione: (2024)

EMC$^2$: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence
di: Yau, Chung-Yiu, et al.
Pubblicazione: (2024)

Adversarial Training Should Be Cast as a Non-Zero-Sum Game
di: Robey, Alexander, et al.
Pubblicazione: (2023)

Randomized algorithms and PAC bounds for inverse reinforcement learning in continuous spaces
di: Kamoutsi, Angeliki, et al.
Pubblicazione: (2024)

Universal Gradient Methods for Stochastic Convex Optimization
di: Rodomanov, Anton, et al.
Pubblicazione: (2024)

On the Role of Batch Size in Stochastic Conditional Gradient Methods
di: Islamov, Rustem, et al.
Pubblicazione: (2026)

Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
di: Oikonomidis, Konstantinos, et al.
Pubblicazione: (2026)

Training Deep Learning Models with Norm-Constrained LMOs
di: Pethick, Thomas, et al.
Pubblicazione: (2025)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)

Adaptive Conditional Gradient Descent
di: Khademi, Abbas, et al.
Pubblicazione: (2025)

Learning to Remove Cuts in Integer Linear Programming
di: Puigdemont, Pol, et al.
Pubblicazione: (2024)

Stochastic Gradient Descent with Adaptive Data
di: Che, Ethan, et al.
Pubblicazione: (2024)

Layer-wise Quantization for Quantized Optimistic Dual Averaging
di: Nguyen, Anh Duc, et al.
Pubblicazione: (2025)

Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexity
di: Yang, Yan, et al.
Pubblicazione: (2024)

Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
di: Glentis, Athanasios, et al.
Pubblicazione: (2025)

First-Order Algorithms Without Lipschitz Gradient: A Sequential Local Optimization Approach
di: Zhang, Junyu, et al.
Pubblicazione: (2020)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
di: Köhne, Frederik, et al.
Pubblicazione: (2023)

EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
di: Yau, Chung-Yiu, et al.
Pubblicazione: (2026)

An Energy-Based Self-Adaptive Learning Rate for Stochastic Gradient Descent: Enhancing Unconstrained Optimization with VAV method
di: Zhang, Jiahao, et al.
Pubblicazione: (2024)

Convergence of Spectral Descent for Non-smooth Optimization
di: Yang, Yixuan, et al.
Pubblicazione: (2026)

Enhancing Fractional Gradient Descent with Learned Optimizers
di: Sobotka, Jan, et al.
Pubblicazione: (2025)

Learning Provably Improves the Convergence of Gradient Descent
di: Song, Qingyu, et al.
Pubblicazione: (2025)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
di: Livni, Roi
Pubblicazione: (2024)

The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization
di: Daskalakis, Constantinos, et al.
Pubblicazione: (2018)

GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms
di: Eleh, Chinedu, et al.
Pubblicazione: (2024)

Towards Guided Descent: Optimization Algorithms for Training Neural Networks At Scale
di: Nagwekar, Ansh
Pubblicazione: (2025)

When Descent Is Too Stable: Event-Triggered Hamiltonian Learning to Optimize
di: Wang, Yi, et al.
Pubblicazione: (2026)

Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization
di: Lin, Tianyi, et al.
Pubblicazione: (2024)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
di: Gautam, Tanmay, et al.
Pubblicazione: (2024)

A Mirror Descent Perspective of Smoothed Sign Descent
di: Wang, Shuyang, et al.
Pubblicazione: (2024)

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
di: Veprikov, Andrey, et al.
Pubblicazione: (2025)

Assortment Optimization for Patient-Provider Matching
di: Raman, Naveen, et al.
Pubblicazione: (2025)

Adaptive Accelerated Gradient Descent Methods for Convex Optimization
di: Xu, Zeyi, et al.
Pubblicazione: (2026)

Zeroth-Order Stochastic Mirror Descent Algorithms for Minimax Excess Risk Optimization
di: Gu, Zhihao, et al.
Pubblicazione: (2024)