Cover Image

Why Adam Works Better with $β_1 = β_2$: The Missing Gradient Scale Invariance Principle

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fernández-Hernández, Alberto, Pérez-Corral, Cristian, Mestre, Jose I., Dolz, Manuel F., Quintana-Ortí, Enrique S.
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Refresh-Scaling the Memory of Balanced Adam
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

StableGrad: Backward Scale Control without Batch Normalization
by: Mestre, Jose I., et al.
Published: (2026)

$λ$-GELU: Learning Gating Hardness for Controlled ReLU-ization in Deep Networks
by: Pérez-Corral, Cristian, et al.
Published: (2026)

OUIDecay: Adaptive Layer-wise Weight Decay for CNNs Using Online Activation Patterns
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

FedOUI: OUI-Guided Client Weighting for Federated Aggregation
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

When Learning Rates Go Wrong: Early Structural Signals in PPO Actor-Critic
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

GLAI: GreenLightningAI for Accelerated Training through Knowledge Decoupling
by: Mestre, Jose I., et al.
Published: (2025)

Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training
by: Pérez-Corral, Cristian, et al.
Published: (2026)

OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
by: Fernández-Hernández, Alberto, et al.
Published: (2026)

FedSQ: Optimized Weight Averaging via Fixed Gating
by: Pérez-Corral, Cristian, et al.
Published: (2026)

Detecting Atypical Clients in Federated Learning via Representation-Level Divergence
by: Pérez-Corral, Cristian, et al.
Published: (2026)

OUI Need to Talk About Weight Decay: A New Perspective on Overfitting Detection
by: Fernández-Hernández, Alberto, et al.
Published: (2025)

Sinusoidal Initialization, Time for a New Start
by: Fernández-Hernández, Alberto, et al.
Published: (2025)

Mapping Parallel Matrix Multiplication in GotoBLAS2 to the AMD Versal ACAP for Deep Learning
by: Lei, Jie, et al.
Published: (2024)

Performance Analysis of Matrix Multiplication for Deep Learning on the Edge
by: Ramírez, Cristian, et al.
Published: (2024)

How to Set $β_1, β_2$ in Adam: An Online Learning Perspective
by: Nguyen, Quan
Published: (2025)

Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
by: Kunstner, Frederik, et al.
Published: (2024)

Fast Truncated SVD of Sparse and Dense Matrices on Graphics Processors
by: Tomas, Andres E., et al.
Published: (2024)

Implementing Basic Arithmetic in $\mathbb{F}_p$ via $\mathbb{F}_2$, and Its Application for Computing the Hamming Distance of Linear Codes
by: Hernando, Fernando, et al.
Published: (2026)

ADOPT: Modified Adam Can Converge with Any $β_2$ with the Optimal Rate
by: Taniguchi, Shohei, et al.
Published: (2024)

Coastal salina evaporites of the Triassic-Liassic boundary in the Iberian Peninsula: the Alacón borehole
by: F. Ortí
Published: (2004)

Invariance Principle and dynamics
by: Ledrappier, François
Published: (2026)

Better Embeddings with Coupled Adam
by: Stollenwerk, Felix, et al.
Published: (2025)

DMRlib: Easy-coding and Efficient Resource Management for Job Malleability
by: Iserte, Sergio, et al.
Published: (2026)

The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference
by: Martínez, Héctor, et al.
Published: (2025)

Taxa de contaminação de testes hematológicos e seus fatores determinantes
by: José Enrique De La Rubia-Ortí
Published: (2014)

Fast Algorithms and Implementations for Computing the Minimum Distance of Quantum Codes
by: Hernando, Fernando, et al.
Published: (2024)

Causal Learning with the Invariance Principle
by: Montagna, Francesco, et al.
Published: (2026)

Multi-environment Invariance Learning with Missing Data
by: Jia, Yiran, et al.
Published: (2026)

Exploiting nested task-parallelism in the $\mathcal{H}-LU$ factorization
by: Carratalá-Sáez, Rocío, et al.
Published: (2019)

La metodología multicriterial y los métodos de valoración de impactos ambientales
by: Serafín Corral Quintana
Published: (2007)

Leadership in the face of digital transformation in an Ecuadorian manufacturing company in 2020
by: Carlos Alberto Ortí Maldonado
Published: (2022)

PREVALENCIA DE TRIPANOSOMIASIS AMERICANA EN MUJERES GESTANTES DE UN ÁREA DE SALUD. VALENCIA, 2005-2007
by: Rafael Manuel Ortí Lucas
Published: (2009)

HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization
by: Huang, Feihu, et al.
Published: (2026)

Boosting the Local Invariance for Better Adversarial Transferability
by: Liu, Bohan, et al.
Published: (2025)

An Invariance Principle for the Multi-slice, with Applications
by: Braverman, Mark, et al.
Published: (2021)

On the Identifiability of Causal Graphs with the Invariance Principle
by: Montagna, Francesco
Published: (2025)

Degrees in the $β$- and $β'$-Delaunay graphs
by: Bonnet, Gilles, et al.
Published: (2025)

Fast Algorithms for the Computation of the Minimum Distance of a Random Linear Code
by: Hernando, Fernando, et al.
Published: (2016)

Scale Invariance, Variety and Central Configurations
by: Lourenço, Maria I. R., et al.
Published: (2026)