Copertina

The Effect of Mini-Batch Noise on the Implicit Bias of Adam

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cattaneo, Matias D., Shigida, Boris
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence Optimization and Control Computation
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On the Implicit Bias of Adam
di: Cattaneo, Matias D., et al.
Pubblicazione: (2023)

How Memory in Optimization Algorithms Implicitly Modifies the Loss
di: Cattaneo, Matias D., et al.
Pubblicazione: (2025)

Modified Loss of Momentum Gradient Descent: Fine-Grained Analysis
di: Cattaneo, Matias D., et al.
Pubblicazione: (2025)

Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
di: Baek, Beomhan, et al.
Pubblicazione: (2025)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
di: Vasudeva, Bhavya, et al.
Pubblicazione: (2025)

A Rod Flow Model for Adam at the Edge of Stability
di: Regis, Eric, et al.
Pubblicazione: (2026)

Optimizer-Induced Mode Connectivity: From AdamW to Muon
di: Zhang, Fangzhao, et al.
Pubblicazione: (2026)

Muon Outperforms Adam in Tail-End Associative Memory Learning
di: Wang, Shuche, et al.
Pubblicazione: (2025)

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization
di: Xie, Shuo, et al.
Pubblicazione: (2024)

How Does Critical Batch Size Scale in Pre-training?
di: Zhang, Hanlin, et al.
Pubblicazione: (2024)

AdamZ: An Enhanced Optimisation Method for Neural Network Training
di: Zaznov, Ilia, et al.
Pubblicazione: (2024)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
di: Meterez, Alexandru, et al.
Pubblicazione: (2025)

Self-Certifying Primal-Dual Optimization Proxies for Large-Scale Batch Economic Dispatch
di: Klamkin, Michael, et al.
Pubblicazione: (2025)

The Implicit Curriculum: Learning Dynamics in RL with Verifiable Rewards
di: Huang, Yu, et al.
Pubblicazione: (2026)

Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
di: Sheen, Heejune, et al.
Pubblicazione: (2024)

Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradients
di: Alvo, Matias, et al.
Pubblicazione: (2026)

Asymptotic and Finite Sample Analysis of Nonexpansive Stochastic Approximations with Markovian Noise
di: Blaser, Ethan, et al.
Pubblicazione: (2024)

Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching
di: Attia, Amit, et al.
Pubblicazione: (2024)

Correlated Noise Provably Beats Independent Noise for Differentially Private Learning
di: Choquette-Choo, Christopher A., et al.
Pubblicazione: (2023)

Nonlinear Non-Gaussian Density Steering with Input and Noise Channel Mismatch: Sinkhorn with Memory for Solving the Control-affine Schrödinger Bridge Problem
di: Bondar, Georgiy A., et al.
Pubblicazione: (2026)

DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization
di: Leenders, Nick, et al.
Pubblicazione: (2025)

Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error
di: Farnia, Farzan, et al.
Pubblicazione: (2026)

Reward Collapse in Aligning Large Language Models
di: Song, Ziang, et al.
Pubblicazione: (2023)

Pinet: Optimizing hard-constrained neural networks with orthogonal projection layers
di: Grontas, Panagiotis D., et al.
Pubblicazione: (2025)

Implicit Bias of Mirror Flow on Separable Data
di: Pesme, Scott, et al.
Pubblicazione: (2024)

The Algorithm Configuration Problem
di: Iommazzo, Gabriele, et al.
Pubblicazione: (2024)

Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
di: Sheng, Jiayuan, et al.
Pubblicazione: (2025)

Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
di: Zhao, Hanyang, et al.
Pubblicazione: (2025)

How Well Can Transformers Emulate In-context Newton's Method?
di: Giannou, Angeliki, et al.
Pubblicazione: (2024)

Stronger Approximation Guarantees for Non-Monotone γ-Weakly DR-Submodular Maximization
di: Jadav, Hareshkumar, et al.
Pubblicazione: (2026)

When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds
di: Tao, Hongyi, et al.
Pubblicazione: (2026)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
di: Fernando, Heshan, et al.
Pubblicazione: (2024)

Solving General Natural-Language-Description Optimization Problems with Large Language Models
di: Zhang, Jihai, et al.
Pubblicazione: (2024)

One-Shot Safety Alignment for Large Language Models via Optimal Dualization
di: Huang, Xinmeng, et al.
Pubblicazione: (2024)

Causal LLM Routing: End-to-End Regret Minimization from Observational Data
di: Tsiourvas, Asterios, et al.
Pubblicazione: (2025)

Reinforcement Learning from Human Feedback with Active Queries
di: Ji, Kaixuan, et al.
Pubblicazione: (2024)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
di: Gautam, Tanmay, et al.
Pubblicazione: (2024)

Algorithmic Challenges in Ensuring Fairness at the Time of Decision
di: Salem, Jad, et al.
Pubblicazione: (2021)

Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
di: Chen, Siyu, et al.
Pubblicazione: (2024)

Variational Learning is Effective for Large Deep Networks
di: Shen, Yuesong, et al.
Pubblicazione: (2024)