Cover Image

Optimization Insights into Deep Diagonal Linear Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Labarrière, Hippolyte, Molinari, Cesare, Rosasco, Lorenzo, Vega, Cristian, Villa, Silvia
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
by: Labarrière, Hippolyte, et al.
Published: (2026)

Stochastic Zeroth order Descent with Structured Directions
by: Rando, Marco, et al.
Published: (2022)

Convergence of zeroth-order proximal point algorithms in the high-temperature regime
by: Naldi, Emanuele, et al.
Published: (2026)

A Structured Tour of Optimization with Finite Differences
by: Rando, Marco, et al.
Published: (2025)

Linear quadratic control of nonlinear systems with Koopman operator learning and the Nyström method
by: Caldarelli, Edoardo, et al.
Published: (2024)

Iterative regularization in classification via hinge loss diagonal descent
by: Apidopoulos, Vassilis, et al.
Published: (2022)

Proximal basin hopping: global optimization with guarantees
by: Lauga, Guillaume, et al.
Published: (2026)

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
by: Papazov, Hristo, et al.
Published: (2024)

High-dimensional Limit of SGD for Diagonal Linear Networks
by: Malaxechebarría, Begoña García, et al.
Published: (2026)

Strong Convergence of FISTA Iterates under H{ö}lderian and Quadratic Growth Conditions
by: Aujol, Jean-François, et al.
Published: (2024)

A Structured Proximal Stochastic Variance Reduced Zeroth-order Algorithm
by: Rando, Marco, et al.
Published: (2025)

Model Consistency of the Iterative Regularization of Dual Ascent for Low-Complexity Regularization
by: Gao, Jie, et al.
Published: (2025)

Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks
by: Qin, Zhen, et al.
Published: (2023)

Dynamic robotic cloth folding with efficient Koopman operator-based model predictive control
by: Caldarelli, Edoardo, et al.
Published: (2026)

Learning Multi-Index Models with Hyper-Kernel Ridge Regression
by: Huang, Shuo, et al.
Published: (2025)

Efficient Sparse PCA via Block-Diagonalization
by: Del Pia, Alberto, et al.
Published: (2024)

SGD with Partial Hessian for Deep Neural Networks Optimization
by: Sun, Ying, et al.
Published: (2024)

On Learning the Optimal Regularization Parameter in Inverse Problems
by: Rodriguez, Jonathan Chirinos, et al.
Published: (2023)

Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis
by: Gomes, Damien Martins
Published: (2025)

Counterfactual Explanations for Linear Optimization
by: Kurtz, Jannis, et al.
Published: (2024)

Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
by: Garrod, Connall, et al.
Published: (2025)

Diagonal Linear Networks and the Lasso Regularization Path
by: Berthier, Raphaël
Published: (2025)

Variance reduction techniques for stochastic proximal point algorithms
by: Traoré, Cheik, et al.
Published: (2023)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
by: Laus, Hannah, et al.
Published: (2025)

Snacks: a fast large-scale kernel SVM solver
by: Tanji, Sofiane, et al.
Published: (2023)

Accelerated Optimization Landscape of Linear-Quadratic Regulator
by: Feng, Lechen, et al.
Published: (2023)

First-Order Methods for Linearly Constrained Bilevel Optimization
by: Kornowski, Guy, et al.
Published: (2024)

Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints
by: Sun, Youbang, et al.
Published: (2024)

Learning to Stop: Deep Learning for Mean Field Optimal Stopping
by: Magnino, Lorenzo, et al.
Published: (2024)

Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks
by: Beneventano, Pierfrancesco, et al.
Published: (2025)

Optimistic Safety for Online Convex Optimization with Unknown Linear Constraints
by: Hutchinson, Spencer, et al.
Published: (2024)

Achieving Linear Speedup with ProxSkip in Distributed Stochastic Optimization
by: Guo, Luyao, et al.
Published: (2023)

PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
by: Li, Bingheng, et al.
Published: (2024)

Active Learning For Contextual Linear Optimization: A Margin-Based Approach
by: Liu, Mo, et al.
Published: (2023)

What Data Enables Optimal Decisions? An Exact Characterization for Linear Optimization
by: Bennouna, Omar, et al.
Published: (2025)

Smart Surrogate Losses for Contextual Stochastic Linear Optimization with Robust Constraints
by: Im, Hyungki, et al.
Published: (2025)

Physics-Informed Neural Networks with Hard Linear Equality Constraints
by: Chen, Hao, et al.
Published: (2024)

Two-Timescale Optimization Framework for Sparse-Feedback Linear-Quadratic Optimal Control
by: Feng, Lechen, et al.
Published: (2024)

Safe and Efficient Online Convex Optimization with Linear Budget Constraints and Partial Feedback
by: Liu, Shanqi, et al.
Published: (2024)

Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraints
by: Huang, Ruichuan, et al.
Published: (2025)