Cover Image

Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jain, Anchit, Nobahari, Rozhin, Baratin, Aristide, Mannelli, Stefano Sarao
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Disordered Systems and Neural Networks
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Bias-inducing geometries: an exactly solvable data model with fairness implications
by: Mannelli, Stefano Sarao, et al.
Published: (2022)

Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks
by: Huang, Jie, et al.
Published: (2026)

The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions
by: Patel, Nishil, et al.
Published: (2023)

The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models
by: Nicoletti, Flavio, et al.
Published: (2026)

Optimal Protocols for Continual Learning via Statistical Physics and Control Theory
by: Mori, Francesco, et al.
Published: (2024)

Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networks
by: Mannelli, Stefano Sarao, et al.
Published: (2024)

Why Do Animals Need Shaping? A Theory of Task Composition and Curriculum Learning
by: Lee, Jin Hwa, et al.
Published: (2024)

Demystifying Spectral Bias on Real-World Data
by: Lavie, Itay, et al.
Published: (2024)

Towards Understanding Inductive Bias in Transformers: A View From Infinity
by: Lavie, Itay, et al.
Published: (2024)

Algorithmic Task Capture, Computational Complexity, and Inductive Bias of Infinite Transformers
by: Davidovich, Orit, et al.
Published: (2026)

Initial Guessing Bias: How Untrained Networks Favor Some Classes
by: Francazi, Emanuele, et al.
Published: (2023)

Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks
by: Arnaboldi, Luca, et al.
Published: (2025)

Grokking as the Transition from Lazy to Rich Training Dynamics
by: Kumar, Tanishq, et al.
Published: (2023)

Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit
by: Meng, Lineghuan, et al.
Published: (2024)

Controlled Langevin Dynamics for Sampling of Feedforward Neural Networks Trained with Minibatches
by: Zambon, Alessandro, et al.
Published: (2026)

Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training
by: Bonnaire, Tony, et al.
Published: (2025)

Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
by: Bordelon, Blake, et al.
Published: (2025)

Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
by: Bocchi, Dario, et al.
Published: (2026)

Statistical Advantage of Softmax Attention: Insights from Single-Location Regression
by: Duranthon, O., et al.
Published: (2025)

Training neural networks with structured noise improves classification and generalization
by: Benedetti, Marco, et al.
Published: (2023)

Fisher Information, Training and Bias in Fourier Regression Models
by: Pastori, Lorenzo, et al.
Published: (2025)

Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breaking
by: Toledo-Marin, J. Quetzalcóatl, et al.
Published: (2025)

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
by: Tanner, Kasimir, et al.
Published: (2024)

The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
by: Mao, Jialin, et al.
Published: (2023)

Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theory
by: Nishiyama, Sota, et al.
Published: (2025)

Dynamical Regimes of Multimodal Diffusion Models
by: Albrychiewicz, Emil, et al.
Published: (2026)

Infinite Limits of Multi-head Transformer Dynamics
by: Bordelon, Blake, et al.
Published: (2024)

A Dynamical Model of Neural Scaling Laws
by: Bordelon, Blake, et al.
Published: (2024)

Geometric Dynamics of Signal Propagation Predict Trainability of Transformers
by: Cowsik, Aditya, et al.
Published: (2024)

Dynamics of Meta-learning Representation in the Teacher-student Scenario
by: Wang, Hui, et al.
Published: (2024)

Dynamical Mean-Field Theory of Self-Attention Neural Networks
by: Poc-López, Ángel, et al.
Published: (2024)

Growing Neural Networks: Dynamic Evolution through Gradient Descent
by: Radhakrishnan, Anil, et al.
Published: (2025)

Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks
by: Montanari, Andrea, et al.
Published: (2025)

Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learning
by: Bordelon, Blake, et al.
Published: (2026)

Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
by: Atanasov, Alexander, et al.
Published: (2025)

Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features
by: Veiga, Rodrigo, et al.
Published: (2024)

High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory
by: Nishiyama, Sota, et al.
Published: (2026)

Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics
by: Avidan, Yehonatan, et al.
Published: (2023)

Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers
by: Mainali, Nischal, et al.
Published: (2025)

A Scalable Measure of Loss Landscape Curvature for Analyzing the Training Dynamics of LLMs
by: Kalra, Dayal Singh, et al.
Published: (2026)