Cover Image

Learning to Forget: Continual Learning with Adaptive Weight Decay

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ramesh, Aditya A., Lewandowski, Alex, Schmidhuber, Jürgen
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discovery
by: Gopalakrishnan, Anand, et al.
Published: (2024)

Who invented deep residual learning?
by: Schmidhuber, Juergen
Published: (2025)

Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning
by: Frati, Lapo, et al.
Published: (2023)

Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion
by: Jaziri, Achref, et al.
Published: (2024)

SNAP: Stopping Catastrophic Forgetting in Hebbian Learning with Sigmoidal Neuronal Adaptive Plasticity
by: Xu, Tianyi, et al.
Published: (2024)

Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning
by: Jaziri, Achref, et al.
Published: (2023)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
by: Csordás, Róbert, et al.
Published: (2023)

A Parameter-free Adaptive Resonance Theory-based Topological Clustering Algorithm Capable of Continual Learning
by: Masuyama, Naoki, et al.
Published: (2023)

The Space Between: On Folding, Symmetries and Sampling
by: Lewandowski, Michal, et al.
Published: (2025)

Annotated History of Modern AI and Deep Learning
by: Schmidhuber, Juergen
Published: (2022)

Deep Learning: Our Miraculous Year 1990-1991
by: Schmidhuber, Juergen
Published: (2020)

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport
by: Cheon, Jeonghwan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

Bio-Inspired Adaptive Neurons for Dynamic Weighting in Artificial Neural Networks
by: Islam, Ashhadul, et al.
Published: (2024)

Dynamically Weighted Momentum with Adaptive Step Sizes for Efficient Deep Network Training
by: Wang, Zhifeng, et al.
Published: (2025)

Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
by: Verma, Lucky
Published: (2026)

Evolutionary Warm-Starts for Reinforcement Learning in Industrial Continuous Control
by: Maus, Tom, et al.
Published: (2026)

Vertical Federated Continual Learning via Evolving Prototype Knowledge
by: Wang, Shuo, et al.
Published: (2025)

Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks
by: Miao, Jiaxing, et al.
Published: (2024)

NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
by: Raghunathan, Karthik Charan, et al.
Published: (2026)

APALU: A Trainable, Adaptive Activation Function for Deep Learning Networks
by: Subramanian, Barathi, et al.
Published: (2024)

A Backpropagation-Free Feedback-Hebbian Network for Continual Learning Dynamics
by: Li, Josh, et al.
Published: (2026)

MoEUT: Mixture-of-Experts Universal Transformers
by: Csordás, Róbert, et al.
Published: (2024)

STAL: Spike Threshold Adaptive Learning Encoder for Classification of Pain-Related Biosignal Data
by: Hens, Freek, et al.
Published: (2024)

ABG-NAS: Adaptive Bayesian Genetic Neural Architecture Search for Graph Representation Learning
by: Wang, Sixuan, et al.
Published: (2025)

MPCS: Neuroplastic Continual Learning via Multi-Component Plasticity and Topology-Aware EWC
by: Hentsch, Joern
Published: (2026)

Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps
by: Vaidya, Hitesh, et al.
Published: (2024)

Towards Constraint-Based Adaptive Hypergraph Learning for Solving Vehicle Routing: An End-to-End Solution
by: Wang, Zhenwei, et al.
Published: (2025)

CantorNet: A Sandbox for Testing Geometrical and Topological Complexity Measures
by: Lewandowski, Michal, et al.
Published: (2024)

Boosting Brain-inspired Path Integration Efficiency via Learning-based Replication of Continuous Attractor Neurodynamics
by: Ge, Zhangyu, et al.
Published: (2025)

Directly Learning Stock Trading Strategies Through Profit Guided Loss Functions
by: Kar, Devroop, et al.
Published: (2025)

Improving Language Plasticity via Pretraining with Active Forgetting
by: Chen, Yihong, et al.
Published: (2023)

Decoupled Weight Decay for Any $p$ Norm
by: Outmezguine, Nadav Joseph, et al.
Published: (2024)

Looped Transformers are Better at Learning Learning Algorithms
by: Yang, Liu, et al.
Published: (2023)

Expressivity of Neural Networks with Random Weights and Learned Biases
by: Williams, Ezekiel, et al.
Published: (2024)

Learning Discretized Bayesian Networks with GOMEA
by: Ha, Damy M. F., et al.
Published: (2024)

Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning
by: Raymond, Christian, et al.
Published: (2022)

FAGH: Accelerating Federated Learning with Approximated Global Hessian
by: Sen, Mrinmay, et al.
Published: (2024)

Gated Recurrent Neural Networks with Weighted Time-Delay Feedback
by: Erichson, N. Benjamin, et al.
Published: (2022)

Learning by the F-adjoint
by: Boughammoura, Ahmed
Published: (2024)