Cover Image

Adam Simplified: Bias Correction Debunked

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Laing, Sam, Orvieto, Antonio
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

In Search of Adam's Secret Sauce
by: Orvieto, Antonio, et al.
Published: (2025)

Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling
by: Srećković, Teodora, et al.
Published: (2025)

Explaining Grokking in Transformers through the Lens of Inductive Bias
by: Singh, Jaisidh, et al.
Published: (2026)

Revisiting associative recall in modern recurrent models
by: Okpekpe, Destiny, et al.
Published: (2025)

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes
by: Orvieto, Antonio, et al.
Published: (2024)

The Implicit Bias of Adam on Separable Data
by: Zhang, Chenyang, et al.
Published: (2024)

DP-AdamW: Investigating Decoupled Weight Decay and Bias Correction in Private Deep Learning
by: Chooi, Jay, et al.
Published: (2025)

Recurrent neural networks: vanishing and exploding gradients are not the end of the story
by: Zucchet, Nicolas, et al.
Published: (2024)

An Uncertainty Principle for Linear Recurrent Neural Networks
by: François, Alexandre, et al.
Published: (2025)

When, Where and Why to Average Weights?
by: Ajroldi, Niccolò, et al.
Published: (2025)

Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers
by: Belloni, Annalisa, et al.
Published: (2026)

Improved state mixing in higher-order and block diagonal linear recurrent networks
by: Dubinin, Igor, et al.
Published: (2026)

On the Implicit Bias of Adam
by: Cattaneo, Matias D., et al.
Published: (2023)

Debunk the Myth of SFT Generalization
by: Lin, Xiaofeng, et al.
Published: (2025)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
by: Vasudeva, Bhavya, et al.
Published: (2025)

The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks
by: Gronich, Eitan, et al.
Published: (2026)

Towards Understanding Self-Pretraining for Sequence Classification
by: Coser, Omar, et al.
Published: (2026)

Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture
by: Movahedi, Sajad, et al.
Published: (2024)

Super Consistency of Neural Network Landscapes and Learning Rate Transfer
by: Noci, Lorenzo, et al.
Published: (2024)

NIMBA: Towards Robust and Principled Processing of Point Clouds With SSMs
by: Köprücü, Nursena, et al.
Published: (2024)

On the low-shot transferability of [V]-Mamba
by: Misra, Diganta, et al.
Published: (2024)

Fixed-Point RNNs: Interpolating from Diagonal to Dense
by: Movahedi, Sajad, et al.
Published: (2025)

Can you Finetune your Binoculars? Embedding Text Watermarks into the Weights of Large Language Models
by: Elhassan, Fay, et al.
Published: (2025)

Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
by: Islamov, Rustem, et al.
Published: (2025)

Loss Landscape Characterization of Neural Networks without Over-Parametrization
by: Islamov, Rustem, et al.
Published: (2024)

Muown: Row-Norm Control for Muon Optimization
by: Lion, Kai, et al.
Published: (2026)

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization
by: Xie, Shuo, et al.
Published: (2024)

Recurrent Distance Filtering for Graph Representation Learning
by: Ding, Yuhui, et al.
Published: (2023)

The Effect of Mini-Batch Noise on the Implicit Bias of Adam
by: Cattaneo, Matias D., et al.
Published: (2026)

GASP: Guided Asymmetric Self-Play For Coding LLMs
by: Jana, Swadesh, et al.
Published: (2026)

Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
by: Meng, Si Yi, et al.
Published: (2025)

Design Principles for Sequence Models via Coefficient Dynamics
by: Sieber, Jerome, et al.
Published: (2025)

The Sweet Danger of Sugar: Debunking Representation Learning for Encrypted Traffic Classification
by: Zhao, Yuqi, et al.
Published: (2025)

On Early-stage Debunking Rumors on Twitter: Leveraging the Wisdom of Weak Learners
by: Nguyen, Tu, et al.
Published: (2017)

Analytical Correction for Subsampling Bias in Drifting Models
by: Zhang, Jiaru, et al.
Published: (2026)

Scaling Behavior of Discrete Diffusion Language Models
by: von Rütte, Dimitri, et al.
Published: (2025)

Generalized Linear Mode Connectivity for Transformers
by: Theus, Alexander, et al.
Published: (2025)

(Almost) Free Modality Stitching of Foundation Models
by: Singh, Jaisidh, et al.
Published: (2025)

CaAdam: Improving Adam optimizer using connection aware methods
by: Genet, Remi, et al.
Published: (2024)

Simplifying Transformer Blocks
by: He, Bobby, et al.
Published: (2023)