Image de couverture de livre

Why Do We Need Weight Decay in Modern Deep Learning?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	D'Angelo, Francesco, Andriushchenko, Maksym, Varre, Aditya, Flammarion, Nicolas
Format:	Preprint
Publié:	2023
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Does Refusal Training in LLMs Generalize to the Past Tense?
par: Andriushchenko, Maksym, et autres
Publié: (2024)

Learning In-context n-grams with Transformers: Sub-n-grams Are Near-stationary Points
par: Varre, Aditya, et autres
Publié: (2025)

Transformers Learn Latent Mixture Models In-Context via Mirror Descent
par: D'Angelo, Francesco, et autres
Publié: (2026)

(How) Learning Rates Regulate Catastrophic Overtraining
par: Rofin, Mark, et autres
Publié: (2026)

Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
par: Andriushchenko, Maksym, et autres
Publié: (2024)

Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions
par: Varre, Aditya, et autres
Publié: (2026)

Is In-Context Learning Sufficient for Instruction Following in LLMs?
par: Zhao, Hao, et autres
Publié: (2024)

Exact Learning of Arithmetic with Differentiable Agents
par: Papazov, Hristo, et autres
Publié: (2025)

Selective Induction Heads: How Transformers Select Causal Structures In Context
par: D'Angelo, Francesco, et autres
Publié: (2025)

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
par: Kuntz, Thomas, et autres
Publié: (2025)

Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs
par: Rando, Javier, et autres
Publié: (2024)

QuantSightBench: Evaluating LLM Quantitative Forecasting with Prediction Intervals
par: Qin, Jeremy, et autres
Publié: (2026)

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning
par: Zhao, Hao, et autres
Publié: (2024)

Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections
par: Schmotz, David, et autres
Publié: (2025)

Why Do We Need Warm-up? A Theoretical Perspective
par: Alimisis, Foivos, et autres
Publié: (2025)

Penalising the biases in norm regularisation enforces sparsity
par: Boursier, Etienne, et autres
Publié: (2023)

Early alignment in two-layer networks training is a two-edged sword
par: Boursier, Etienne, et autres
Publié: (2024)

Simplicity bias and optimization threshold in two-layer ReLU networks
par: Boursier, Etienne, et autres
Publié: (2024)

Learning Parametric Distributions from Samples and Preferences
par: Jourdan, Marc, et autres
Publié: (2025)

HalluHard: A Hard Multi-Turn Hallucination Benchmark
par: Fan, Dongyang, et autres
Publié: (2026)

Learning to Forget: Continual Learning with Adaptive Weight Decay
par: Ramesh, Aditya A., et autres
Publié: (2026)

JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
par: Chao, Patrick, et autres
Publié: (2024)

Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data
par: Salerno, Stephen, et autres
Publié: (2025)

Learning Algorithms in the Limit
par: Papazov, Hristo, et autres
Publié: (2025)

Critical Influence of Overparameterization on Sharpness-aware Minimization
par: Shin, Sungbin, et autres
Publié: (2023)

Exploring Memorization and Copyright Violation in Frontier LLMs: A Study of the New York Times v. OpenAI 2023 Lawsuit
par: Freeman, Joshua, et autres
Publié: (2024)

Layer-wise Linear Mode Connectivity
par: Adilova, Linara, et autres
Publié: (2023)

Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
par: Schmotz, David, et autres
Publié: (2026)

Do We Really Even Need Data?
par: Hoffman, Kentaro, et autres
Publié: (2024)

FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens
par: Schlarmann, Christian, et autres
Publié: (2025)

On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks
par: Neuhaus, Yannic, et autres
Publié: (2026)

Incremental Learning of Sparse Attention Patterns in Transformers
par: Yüksel, Oğuz Kaan, et autres
Publié: (2026)

Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?
par: Zhang, Kaiqi, et autres
Publié: (2022)

First-order ANIL provably learns representations despite overparametrization
par: Yüksel, Oğuz Kaan, et autres
Publié: (2023)

Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
par: Boursier, Etienne, et autres
Publié: (2022)

Capability-Based Scaling Trends for LLM-Based Red-Teaming
par: Panfilov, Alexander, et autres
Publié: (2025)

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
par: Papazov, Hristo, et autres
Publié: (2024)

Do We Need Transformers to Play FPS Video Games?
par: Batth, Karmanbir, et autres
Publié: (2025)

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents
par: Talokar, Nivya, et autres
Publié: (2026)

Deep Language Geometry: Constructing a Metric Space from LLM Weights
par: Shamrai, Maksym, et autres
Publié: (2025)