Cover Image

Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tsilivis, Nikolaos, Gronich, Eitan, Kempe, Julia, Vardi, Gal
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks
by: Gronich, Eitan, et al.
Published: (2026)

The Price of Implicit Bias in Adversarially Robust Generalization
by: Tsilivis, Nikolaos, et al.
Published: (2024)

On the Robustness of Neural Collapse and the Neural Collapse of Robustness
by: Su, Jingtong, et al.
Published: (2023)

How Reinforcement Learning After Next-Token Prediction Facilitates Learning
by: Tsilivis, Nikolaos, et al.
Published: (2025)

The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient
by: Li, Jichu, et al.
Published: (2026)

Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks
by: Feng, Yunzhen, et al.
Published: (2024)

Learning to Think from Multiple Thinkers
by: Joshi, Nirmit, et al.
Published: (2026)

On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds
by: Vilucchio, Matteo, et al.
Published: (2024)

Implicit Regularization Towards Rank Minimization in ReLU Networks
by: Timor, Nadav, et al.
Published: (2022)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
by: Cai, Yuhang, et al.
Published: (2025)

ImpMIA: Leveraging Implicit Bias for Membership Inference Attack
by: Golbari, Yuval, et al.
Published: (2025)

Faster Acceleration for Steepest Descent
by: Bai, Cedar Site, et al.
Published: (2024)

Provable Privacy Attacks on Trained Shallow Neural Networks
by: Smorodinsky, Guy, et al.
Published: (2024)

Never Saddle for Reparameterized Steepest Descent as Mirror Flow
by: Jacobs, Tom, et al.
Published: (2026)

Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks
by: Li, Binghui, et al.
Published: (2024)

Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
by: Jacobs, Tom, et al.
Published: (2026)

Provable Unlearning with Gradient Ascent on Two-Layer ReLU Neural Networks
by: Melamed, Odelia, et al.
Published: (2025)

Noisy Interpolation Learning with Shallow Univariate ReLU Networks
by: Joshi, Nirmit, et al.
Published: (2023)

Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context
by: Frei, Spencer, et al.
Published: (2024)

Transformers are almost optimal metalearners for linear classification
by: Magen, Roey, et al.
Published: (2025)

The Implicit Bias of Gradient Descent on Separable Data
by: Soudry, Daniel, et al.
Published: (2017)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
by: Vaswani, Sharan, et al.
Published: (2026)

Generalization Bounds of Stochastic Gradient Descent in Homogeneous Neural Networks
by: Ma, Wenquan, et al.
Published: (2026)

Deconstructing the Goldilocks Zone of Neural Network Initialization
by: Vysogorets, Artem, et al.
Published: (2024)

The Implicit Bias of Gradient Descent on Separable Multiclass Data
by: Ravi, Hrithik, et al.
Published: (2024)

Querying Kernel Methods Suffices for Reconstructing their Training Data
by: Barzilai, Daniel, et al.
Published: (2025)

SPGD: Steepest Perturbed Gradient Descent Optimization
by: Vahedi, Amir M., et al.
Published: (2024)

Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality
by: Medvedev, Marko, et al.
Published: (2024)

To Grok Grokking: Provable Grokking in Ridge Regression
by: Xu, Mingyue, et al.
Published: (2026)

Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
by: Akhtiamov, Danil, et al.
Published: (2026)

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
by: Cai, Yuhang, et al.
Published: (2024)

Stacey: Promoting Stochastic Steepest Descent via Accelerated $\ell_p$-Smooth Nonconvex Optimization
by: Luo, Xinyu, et al.
Published: (2025)

How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
by: Lai, Kuo-Wei, et al.
Published: (2026)

Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
by: Fan, Chen, et al.
Published: (2025)

Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
by: Jung, Hyunji, et al.
Published: (2025)

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
by: Veprikov, Andrey, et al.
Published: (2025)

The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
by: Edelman, Benjamin L., et al.
Published: (2024)

Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks
by: Peleg, Amit, et al.
Published: (2024)

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
by: Vyas, Nikhil, et al.
Published: (2023)

Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression
by: Liang, Shuang, et al.
Published: (2024)