Image de couverture de livre

Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kumar, Akshay, Haupt, Jarvis
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Directional Convergence Near Small Initializations and Saddles in Two-Homogeneous Neural Networks
par: Kumar, Akshay, et autres
Publié: (2024)

Towards Understanding Gradient Flow Dynamics of Homogeneous Neural Networks Beyond the Origin
par: Kumar, Akshay, et autres
Publié: (2025)

Learning Neural Networks by Neuron Pursuit
par: Kumar, Akshay, et autres
Publié: (2025)

Convergence of Gradient Descent with Small Initialization for Unregularized Matrix Completion
par: Ma, Jianhao, et autres
Publié: (2024)

Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks
par: Qin, Zhen, et autres
Publié: (2023)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
par: Liao, Fangshuo, et autres
Publié: (2023)

Deconstructing the Goldilocks Zone of Neural Network Initialization
par: Vysogorets, Artem, et autres
Publié: (2024)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
par: Cai, Yuhang, et autres
Publié: (2025)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
par: Laus, Hannah, et autres
Publié: (2025)

Global Convergence of Four-Layer Matrix Factorization under Random Initialization
par: Luo, Minrui, et autres
Publié: (2025)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
par: Cayci, Semih, et autres
Publié: (2024)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
par: Mishkin, Aaron, et autres
Publié: (2024)

Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
par: Das, Mohua, et autres
Publié: (2026)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
par: Xu, Xianliang, et autres
Publié: (2024)

Robust Angular Synchronization via Directed Graph Neural Networks
par: He, Yixuan, et autres
Publié: (2023)

Convergence of SGD for Training Neural Networks with Sliced Wasserstein Losses
par: Tanguy, Eloi
Publié: (2023)

SGD with Partial Hessian for Deep Neural Networks Optimization
par: Sun, Ying, et autres
Publié: (2024)

Deep Operator Neural Network Model Predictive Control
par: de Jong, Thomas Oliver, et autres
Publié: (2025)

Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning
par: Caron, Francois, et autres
Publié: (2023)

Improved Scalable Lipschitz Bounds for Deep Neural Networks
par: Syed, Usman, et autres
Publié: (2025)

Global Convergence of SGD On Two Layer Neural Nets
par: Gopalani, Pulkit, et autres
Publié: (2022)

Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness
par: Bo, Yanan, et autres
Publié: (2026)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
par: Tucat, Matteo, et autres
Publié: (2024)

Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networks
par: Le, Bach C., et autres
Publié: (2025)

Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling
par: Rosso, Haley, et autres
Publié: (2025)

QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning
par: Fu, Minghan, et autres
Publié: (2023)

Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes
par: Zhang, Erica, et autres
Publié: (2024)

CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural Networks
par: Feng, Miria, et autres
Publié: (2024)

Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
par: Gopalani, Pulkit, et autres
Publié: (2023)

On the Convergence of Overparameterized Problems: Inherent Properties of the Compositional Structure of Neural Networks
par: de Oliveira, Arthur Castello Branco, et autres
Publié: (2025)

An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture
par: Krishnanunni, C G, et autres
Publié: (2022)

Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theory
par: Tabuada, Paulo, et autres
Publié: (2020)

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
par: Cai, Yuhang, et autres
Publié: (2024)

optimizn: a Python Library for Developing Customized Optimization Algorithms
par: Sathiya, Akshay, et autres
Publié: (2025)

ADMM Algorithms for Residual Network Training: Convergence Analysis and Parallel Implementation
par: Xu, Jintao, et autres
Publié: (2023)

BONSAI: Structure-exploiting robust Bayesian optimization for networked black-box systems under uncertainty
par: Kudva, Akshay, et autres
Publié: (2025)

A Homogenization Approach for Gradient-Dominated Stochastic Optimization
par: Tan, Jiyuan, et autres
Publié: (2023)

Weak Convergence Analysis of Online Neural Actor-Critic Algorithms
par: Lam, Samuel Chun-Hei, et autres
Publié: (2024)

Optimization Landscapes Learned: Proxy Networks Boost Convergence in Physics-based Inverse Problems
par: Goyal, Girnar, et autres
Publié: (2025)

Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learning
par: Amortila, Philip, et autres
Publié: (2024)