Image de couverture de livre

Directional Convergence Near Small Initializations and Saddles in Two-Homogeneous Neural Networks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kumar, Akshay, Haupt, Jarvis
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
par: Kumar, Akshay, et autres
Publié: (2024)

Towards Understanding Gradient Flow Dynamics of Homogeneous Neural Networks Beyond the Origin
par: Kumar, Akshay, et autres
Publié: (2025)

Learning Neural Networks by Neuron Pursuit
par: Kumar, Akshay, et autres
Publié: (2025)

Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks
par: Zhang, Leyang, et autres
Publié: (2024)

Convergence of Gradient Descent with Small Initialization for Unregularized Matrix Completion
par: Ma, Jianhao, et autres
Publié: (2024)

Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
par: Bylinkin, Dmitry, et autres
Publié: (2025)

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks
par: Xu, Xianliang, et autres
Publié: (2024)

Global Convergence of SGD On Two Layer Neural Nets
par: Gopalani, Pulkit, et autres
Publié: (2022)

Deconstructing the Goldilocks Zone of Neural Network Initialization
par: Vysogorets, Artem, et autres
Publié: (2024)

On Linear Convergence in Smooth Convex-Concave Bilinearly-Coupled Saddle-Point Optimization: Lower Bounds and Optimal Algorithms
par: Kovalev, Dmitry, et autres
Publié: (2024)

Federated Composite Saddle Point Optimization
par: Bai, Site, et autres
Publié: (2023)

Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
par: Gopalani, Pulkit, et autres
Publié: (2023)

Quantization Avoids Saddle Points in Distributed Optimization
par: Bo, Yanan, et autres
Publié: (2024)

Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks
par: Qin, Zhen, et autres
Publié: (2023)

Global Convergence of Four-Layer Matrix Factorization under Random Initialization
par: Luo, Minrui, et autres
Publié: (2025)

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
par: Cai, Yuhang, et autres
Publié: (2024)

Proximal Point Method for Online Saddle Point Problem
par: Meng, Qing-xin, et autres
Publié: (2024)

Inertial Newton Algorithms Avoiding Strict Saddle Points
par: Castera, Camille
Publié: (2021)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
par: Cayci, Semih, et autres
Publié: (2024)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
par: Mishkin, Aaron, et autres
Publié: (2024)

Simultaneous Learning and Optimization via Misspecified Saddle Point Problems
par: Ahmadi, Mohammad Mahdi, et autres
Publié: (2025)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
par: Liao, Fangshuo, et autres
Publié: (2023)

Mirror Descent Algorithms with Nearly Dimension-Independent Rates for Differentially-Private Stochastic Saddle-Point Problems
par: González, Tomás, et autres
Publié: (2024)

Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points
par: Yamamoto, Naoya, et autres
Publié: (2025)

Robust Angular Synchronization via Directed Graph Neural Networks
par: He, Yixuan, et autres
Publié: (2023)

Convergence of SGD for Training Neural Networks with Sliced Wasserstein Losses
par: Tanguy, Eloi
Publié: (2023)

Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
par: Han, Yinbin, et autres
Publié: (2023)

Nearly Optimal Linear Convergence of Stochastic Primal-Dual Methods for Linear Programming
par: Lu, Haihao, et autres
Publié: (2021)

Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
par: Beznosikov, Aleksandr, et autres
Publié: (2026)

Online Min-Max Optimization: From Individual Regrets to Cumulative Saddle Points
par: Vyas, Abhijeet, et autres
Publié: (2026)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
par: Cai, Yuhang, et autres
Publié: (2025)

Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
par: Cao, Daniel Yiming, et autres
Publié: (2025)

Escaping Saddle Points for Nonsmooth Weakly Convex Functions via Perturbed Proximal Algorithms
par: Huang, Minhui, et autres
Publié: (2021)

Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning
par: Caron, Francois, et autres
Publié: (2023)

Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
par: Beznosikov, Aleksandr, et autres
Publié: (2020)

Convex Formulations for Training Two-Layer ReLU Neural Networks
par: Prakhya, Karthik, et autres
Publié: (2024)

From Saddle Points Toward Global Minima: A Newton-Type Method on Wasserstein Space
par: Lascu, Razvan-Andrei, et autres
Publié: (2026)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
par: Laus, Hannah, et autres
Publié: (2025)

Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness
par: Bo, Yanan, et autres
Publié: (2026)

Near-Optimal Convergence of Accelerated Gradient Methods under Generalized and $(L_0, L_1)$-Smoothness
par: Tyurin, Alexander
Publié: (2025)