Image de couverture de livre

Towards Guided Descent: Optimization Algorithms for Training Neural Networks At Scale

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Nagwekar, Ansh
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning
par: Zhang, Thomas T., et autres
Publié: (2025)

Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks
par: Jnini, Anas, et autres
Publié: (2025)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
par: Xu, Xianliang, et autres
Publié: (2024)

Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization
par: Lin, Tianyi, et autres
Publié: (2024)

Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time
par: Fu, Qiang, et autres
Publié: (2025)

Zeroth-Order Stochastic Mirror Descent Algorithms for Minimax Excess Risk Optimization
par: Gu, Zhihao, et autres
Publié: (2024)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
par: Cayci, Semih, et autres
Publié: (2024)

Optimization Over Trained Neural Networks: Taking a Relaxing Walk
par: Tong, Jiatai, et autres
Publié: (2024)

Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study
par: Chen, Cheng, et autres
Publié: (2020)

Non-Singularity of the Gradient Descent map for Neural Networks with Piecewise Analytic Activations
par: Crăciun, Alexandru, et autres
Publié: (2025)

Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization
par: Zheng, Tianqi, et autres
Publié: (2024)

Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law
par: Kunstner, Frederik, et autres
Publié: (2025)

Homotopy Relaxation Training Algorithms for Infinite-Width Two-Layer ReLU Neural Networks
par: Yang, Yahong, et autres
Publié: (2023)

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization
par: Zhu, Yuchen, et autres
Publié: (2024)

Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise
par: Zhang, Jiayu, et autres
Publié: (2026)

Mean-Field Limits for Two-Layer Neural Networks Trained with Consensus-Based Optimization
par: De Deyn, William, et autres
Publié: (2025)

Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate
par: Pham, Hung, et autres
Publié: (2025)

Enhancing Fractional Gradient Descent with Learned Optimizers
par: Sobotka, Jan, et autres
Publié: (2025)

Convergence of Spectral Descent for Non-smooth Optimization
par: Yang, Yixuan, et autres
Publié: (2026)

Dual Cone Gradient Descent for Training Physics-Informed Neural Networks
par: Hwang, Youngsik, et autres
Publié: (2024)

Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation
par: Kong, Deyi, et autres
Publié: (2026)

Mirror Descent-Type Algorithms for the Variational Inequality Problem with Functional Constraints
par: Alkousa, Mohammad S., et autres
Publié: (2026)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
par: Livni, Roi
Publié: (2024)

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
par: Cai, Yuhang, et autres
Publié: (2024)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
par: Laus, Hannah, et autres
Publié: (2025)

GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms
par: Eleh, Chinedu, et autres
Publié: (2024)

The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization
par: Daskalakis, Constantinos, et autres
Publié: (2018)

MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
par: Ozkara, Kaan, et autres
Publié: (2024)

How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
par: Lai, Kuo-Wei, et autres
Publié: (2026)

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
par: Vaswani, Sharan, et autres
Publié: (2021)

Curse of Dimensionality in Neural Network Optimization
par: Na, Sanghoon, et autres
Publié: (2025)

Stochastic Adaptive Gradient Descent Without Descent
par: Aujol, Jean-François, et autres
Publié: (2025)

Training Neural ODEs Using Fully Discretized Simultaneous Optimization
par: Shapovalova, Mariia, et autres
Publié: (2025)

Relaxation-Informed Training of Neural Network Surrogate Models
par: Tsay, Calvin
Publié: (2026)

Adaptive Momentum and Nonlinear Damping for Neural Network Training
par: Karoni, Aikaterini, et autres
Publié: (2026)

Towards Quantifying the Hessian Structure of Neural Networks
par: Dong, Zhaorui, et autres
Publié: (2025)

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
par: Jacot, Arthur, et autres
Publié: (2024)

ADMM Algorithms for Residual Network Training: Convergence Analysis and Parallel Implementation
par: Xu, Jintao, et autres
Publié: (2023)

When Descent Is Too Stable: Event-Triggered Hamiltonian Learning to Optimize
par: Wang, Yi, et autres
Publié: (2026)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
par: Cai, Yuhang, et autres
Publié: (2025)