Image de couverture de livre

Correction of Decoupled Weight Decay

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Chou, Jason Chuan-Chih
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DP-AdamW: Investigating Decoupled Weight Decay and Bias Correction in Private Deep Learning
par: Chooi, Jay, et autres
Publié: (2025)

Embedding Geometries of Contrastive Language-Image Pre-Training
par: Chou, Jason Chuan-Chih, et autres
Publié: (2024)

ViT Registers and Fractal ViT
par: Chou, Jason Chuan-Chih, et autres
Publié: (2026)

Decoupled Weight Decay for Any $p$ Norm
par: Outmezguine, Nadav Joseph, et autres
Publié: (2024)

Cautious Weight Decay
par: Chen, Lizhang, et autres
Publié: (2025)

From $\log π$ to $π$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
par: Fu, Xiaoliang, et autres
Publié: (2026)

Does Weight Decay Enhance Training Stability?
par: Saether, Marius, et autres
Publié: (2026)

Why Do We Need Weight Decay in Modern Deep Learning?
par: D'Angelo, Francesco, et autres
Publié: (2023)

Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay
par: Pan, Leyan, et autres
Publié: (2023)

SGD and Weight Decay Secretly Minimize the Rank of Your Neural Network
par: Galanti, Tomer, et autres
Publié: (2022)

The Lifecycle of the Spectral Edge: From Gradient Learning to Weight-Decay Compression
par: Xu, Yongzhong
Publié: (2026)

AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
par: He, Di, et autres
Publié: (2025)

Cosine-Gated Adam-Decay: Drop-In Staleness-Aware Outer Optimization for Decoupled DiLoCo
par: Shah, Vatsal, et autres
Publié: (2026)

Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?
par: Zhang, Kaiqi, et autres
Publié: (2022)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
par: Kosson, Atli, et autres
Publié: (2023)

Learning to Forget: Continual Learning with Adaptive Weight Decay
par: Ramesh, Aditya A., et autres
Publié: (2026)

On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective
par: Loveland, Donald, et autres
Publié: (2025)

Robust Layerwise Scaling Rules by Proper Weight Decay Tuning
par: Fan, Zhiyuan, et autres
Publié: (2025)

Weighted Temporal Decay Loss for Learning Wearable PPG Data with Sparse Clinical Labels
par: Chung, Yunsung, et autres
Publié: (2026)

Towards Better Generalization: Weight Decay Induces Low-rank Bias for Neural Networks
par: Chen, Ke, et autres
Publié: (2024)

OUIDecay: Adaptive Layer-wise Weight Decay for CNNs Using Online Activation Patterns
par: Fernández-Hernández, Alberto, et autres
Publié: (2026)

A Self-Attentive Meta-Optimizer with Group-Adaptive Learning Rates and Weight Decay
par: Zhao, JiangBo, et autres
Publié: (2026)

Robust Implicit Regularization via Weight Normalization
par: Chou, Hung-Hsu, et autres
Publié: (2023)

Constraint Learning in Multi-Agent Dynamic Games from Demonstrations of Local Nash Interactions
par: Zhang, Zhouyu, et autres
Publié: (2025)

Decoupled Travel Planning with Behavior Forest
par: Yuan, Duanyang, et autres
Publié: (2026)

Weight Decay may matter more than muP for Learning Rate Transfer in Practice
par: Kosson, Atli, et autres
Publié: (2025)

SG-XDEAT: Sparsity-Guided Cross-Dimensional and Cross-Encoding Attention with Target-Aware Conditioning in Tabular Learning
par: Cheng, Chih-Chuan, et autres
Publié: (2025)

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
par: Jacot, Arthur, et autres
Publié: (2024)

Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
par: Xu, Huangyu, et autres
Publié: (2026)

The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning
par: Wu, Zihao, et autres
Publié: (2026)

Online Imitation Learning for Manipulation via Decaying Relative Correction through Teleoperation
par: Pan, Cheng, et autres
Publié: (2025)

The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure
par: Xu, Yongzhong
Publié: (2026)

On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective
par: Xie, Zeke, et autres
Publié: (2020)

Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
par: Verma, Lucky
Publié: (2026)

Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models
par: Tian, Junjiao, et autres
Publié: (2024)

Tune without Validation: Searching for Learning Rate and Weight Decay on Training Sets
par: Brigato, Lorenzo, et autres
Publié: (2024)

Trainable Weight Averaging: Accelerating Training and Improving Generalization
par: Li, Tao, et autres
Publié: (2022)

Impact of Loss Weight and Model Complexity on Physics-Informed Neural Networks for Computational Fluid Dynamics
par: Chou, Yi En, et autres
Publié: (2025)

AdamHD: Decoupled Huber Decay Regularization for Language Model Pre-Training
par: Guo, Fu-Ming, et autres
Publié: (2025)

Weight-Decay Turns Transformer Loss Landscapes Villani: Functional-Analytic Foundations for Optimization and Generalization
par: Das, Abhijit, et autres
Publié: (2026)