Image de couverture de livre

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Abuduweili, Abulikemu, Liu, Changliu
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Estimating Neural Network Robustness via Lipschitz Constant and Architecture Sensitivity
par: Abuduweili, Abulikemu, et autres
Publié: (2024)

Enhancing Sample Generation of Diffusion Models using Noise Level Correction
par: Abuduweili, Abulikemu, et autres
Publié: (2024)

The Initialization Determines Whether In-Context Learning Is Gradient Descent
par: Xie, Shifeng, et autres
Publié: (2025)

Adaptive Heavy-Tailed Stochastic Gradient Descent
par: Gong, Bodu, et autres
Publié: (2025)

Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent
par: Liu, Weihua, et autres
Publié: (2024)

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization
par: Xu, Mingjing, et autres
Publié: (2024)

Conflict-Averse Gradient Descent for Multi-task Learning
par: Liu, Bo, et autres
Publié: (2021)

Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization
par: Kumar, Ramnath, et autres
Publié: (2023)

Gradient Descent Algorithm Survey
par: Fucheng, Deng, et autres
Publié: (2025)

Improving Policy Optimization via $\varepsilon$-Retrain
par: Marzari, Luca, et autres
Publié: (2024)

Noise Balance and Stationary Distribution of Stochastic Gradient Descent
par: Ziyin, Liu, et autres
Publié: (2023)

Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networks
par: Wang, Puyu, et autres
Publié: (2026)

Learning Associative Memories with Gradient Descent
par: Cabannes, Vivien, et autres
Publié: (2024)

Randomness and Interpolation Improve Gradient Descent
par: Li, Jiawen, et autres
Publié: (2025)

ONG: Orthogonal Natural Gradient Descent
par: Yadav, Yajat, et autres
Publié: (2025)

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
par: Cheng, Wenhua, et autres
Publié: (2023)

Optimizing Predictive AI in Physical Design Flows with Mini Pixel Batch Gradient Descent
par: Yang, Haoyu, et autres
Publié: (2024)

Auto-Unrolled Proximal Gradient Descent: An AutoML Approach to Interpretable Waveform Optimization
par: Kaplan, Ahmet
Publié: (2026)

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
par: Liang, Chen, et autres
Publié: (2026)

Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees
par: Wu, Yongtao, et autres
Publié: (2025)

Vanilla Gradient Descent for Oblique Decision Trees
par: Panda, Subrat Prasad, et autres
Publié: (2024)

Stochastic Gradient Descent with Momentum is Algorithmically Stable
par: Lei, Yunwen, et autres
Publié: (2026)

Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent
par: Naganuma, Hiroki, et autres
Publié: (2026)

Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training
par: Ma, Yuhan, et autres
Publié: (2024)

Grams: Gradient Descent with Adaptive Momentum Scaling
par: Cao, Yang, et autres
Publié: (2024)

Sequential Policy Gradient for Adaptive Hyperparameter Optimization
par: Li, Zheng, et autres
Publié: (2025)

Elastic Multi-Gradient Descent for Parallel Continual Learning
par: Lyu, Fan, et autres
Publié: (2024)

Efficient Search for Customized Activation Functions with Gradient Descent
par: Strack, Lukas, et autres
Publié: (2024)

Can LLMs predict the convergence of Stochastic Gradient Descent?
par: Zekri, Oussama, et autres
Publié: (2024)

FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
par: Liu, Junkang, et autres
Publié: (2026)

Geodesic Gradient Descent: A Generic and Learning-rate-free Optimizer on Objective Function-induced Manifolds
par: Hu, Liwei, et autres
Publié: (2026)

Gradient Descent Efficiency Index
par: Dhingra, Aviral
Publié: (2024)

Absolute Policy Optimization
par: Zhao, Weiye, et autres
Publié: (2023)

Revisiting Glorot Initialization for Long-Range Linear Recurrences
par: Bar, Noga, et autres
Publié: (2025)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
par: Garg, Ishir, et autres
Publié: (2026)

Implicit Safe Set Algorithm for Provably Safe Reinforcement Learning
par: Zhao, Weiye, et autres
Publié: (2024)

Adaptive Online Mirror Descent for Tchebycheff Scalarization in Multi-Objective Learning
par: Liu, Meitong, et autres
Publié: (2024)

Optimizing ML Training with Metagradient Descent
par: Engstrom, Logan, et autres
Publié: (2025)

Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descent
par: Kumar, Mohit, et autres
Publié: (2024)

GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent
par: Marton, Sascha, et autres
Publié: (2023)