Cover Image

DP-AdamW: Investigating Decoupled Weight Decay and Bias Correction in Private Deep Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chooi, Jay, Cong, Kevin, Li, Russell, Sun, Lillian
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization
by: Xie, Shuo, et al.
Published: (2024)

APOLLO: SGD-like Memory, AdamW-level Performance
by: Zhu, Hanqing, et al.
Published: (2024)

Uniform Scaling Limits in AdamW-Trained Transformers
by: Gibson, William, et al.
Published: (2026)

HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization
by: Huang, Feihu, et al.
Published: (2026)

Correction of Decoupled Weight Decay
by: Chou, Jason Chuan-Chih
Published: (2025)

Optimizer-Induced Mode Connectivity: From AdamW to Muon
by: Zhang, Fangzhao, et al.
Published: (2026)

Beyond First-Order: Training LLMs with Stochastic Conjugate Subgradients and AdamW
by: Zhang, Di, et al.
Published: (2025)

How to set AdamW's weight decay as you scale model and dataset size
by: Wang, Xi, et al.
Published: (2024)

Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
by: Ranganath, Aditya
Published: (2026)

Convergence Rate Analysis of the AdamW-Style Shampoo: Unifying One-Sided and Two-Sided Preconditioning
by: Li, Huan, et al.
Published: (2026)

DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models
by: Liu, Jin, et al.
Published: (2026)

First-Passage Prediction of Grokking Delay: ACalibrated Law under AdamW with Causal Validation
by: Khanh, Truong Xuan, et al.
Published: (2026)

On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
by: Li, Huan, et al.
Published: (2025)

DP-MicroAdam: Private and Frugal Algorithm for Training and Fine-tuning
by: Hudişteanu, Mihaela, et al.
Published: (2025)

Cosine-Gated Adam-Decay: Drop-In Staleness-Aware Outer Optimization for Decoupled DiLoCo
by: Shah, Vatsal, et al.
Published: (2026)

Adam Simplified: Bias Correction Debunked
by: Laing, Sam, et al.
Published: (2025)

DP-GPL: Differentially Private Graph Prompt Learning
by: Xu, Jing, et al.
Published: (2025)

Cautious Weight Decay
by: Chen, Lizhang, et al.
Published: (2025)

Decoupled Weight Decay for Any $p$ Norm
by: Outmezguine, Nadav Joseph, et al.
Published: (2024)

Too Good to be True? Turn Any Model Differentially Private With DP-Weights
by: Zagardo, David
Published: (2024)

Why Do We Need Weight Decay in Modern Deep Learning?
by: D'Angelo, Francesco, et al.
Published: (2023)

AdamHD: Decoupled Huber Decay Regularization for Language Model Pre-Training
by: Guo, Fu-Ming, et al.
Published: (2025)

FusionDP: Foundation Model-Assisted Differentially Private Learning for Partially Sensitive Features
by: Zeng, Linghui, et al.
Published: (2025)

FlashDP: Private Training Large Language Models with Efficient DP-SGD
by: Wang, Liangyu, et al.
Published: (2025)

DP-DCAN: Differentially Private Deep Contrastive Autoencoder Network for Single-cell Clustering
by: Li, Huifa, et al.
Published: (2023)

DP-HYPE: Distributed Differentially Private Hyperparameter Search
by: Liebenow, Johannes, et al.
Published: (2025)

From $\log π$ to $π$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
by: Fu, Xiaoliang, et al.
Published: (2026)

FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations
by: Wang, Hui-Po, et al.
Published: (2023)

Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?
by: Zhang, Kaiqi, et al.
Published: (2022)

DP-EMAR: A Differentially Private Framework for Autonomous Model Weight Repair in Federated IoT Systems
by: Kabgere, Chethana Prasad, et al.
Published: (2025)

The Implicit Bias of Adam on Separable Data
by: Zhang, Chenyang, et al.
Published: (2024)

Towards Better Generalization: Weight Decay Induces Low-rank Bias for Neural Networks
by: Chen, Ke, et al.
Published: (2024)

How Private are DP-SGD Implementations?
by: Chua, Lynn, et al.
Published: (2024)

DP-LDMs: Differentially Private Latent Diffusion Models
by: Liu, Michael F., et al.
Published: (2023)

DP-KAN: Differentially Private Kolmogorov-Arnold Networks
by: Kalinin, Nikita P., et al.
Published: (2024)

A Differentially Private Weighted Empirical Risk Minimization Procedure and its Application to Outcome Weighted Learning
by: Giddens, Spencer, et al.
Published: (2023)

DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum
by: Kim, Jihwan, et al.
Published: (2026)

On the Implicit Bias of Adam
by: Cattaneo, Matias D., et al.
Published: (2023)

FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction
by: Dm, Duc, et al.
Published: (2026)

DP-TabICL: In-Context Learning with Differentially Private Tabular Data
by: Carey, Alycia N., et al.
Published: (2024)