Image de couverture de livre

The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Goldfarb, Daniel, Evron, Itay, Weinberger, Nir, Soudry, Daniel, Hand, Paul
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Analysis of Overparameterization in Continual Learning under a Linear Model
par: Goldfarb, Daniel, et autres
Publié: (2025)

Are Greedy Task Orderings Better Than Random in Continual Linear Regression?
par: Tsipory, Matan, et autres
Publié: (2025)

Optimal L2 Regularization in High-dimensional Continual Linear Regression
par: Karpel, Gilad, et autres
Publié: (2026)

From Continual Learning to SGD and Back: Better Rates for Continual Linear Models
par: Evron, Itay, et autres
Publié: (2025)

Provable Tempered Overfitting of Minimal Nets and Typical Nets
par: Harel, Itamar, et autres
Publié: (2024)

Optimal Rates in Continual Linear Regression via Increasing Regularization
par: Levinstein, Ran, et autres
Publié: (2025)

Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks
par: Kinderman, Edan, et autres
Publié: (2024)

How do Minimum-Norm Shallow Denoisers Look in Function Space?
par: Zeno, Chen, et autres
Publié: (2023)

When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets
par: Zeno, Chen, et autres
Publié: (2025)

Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators
par: Blumenfeld, Yaniv, et autres
Publié: (2024)

Minimum Variance Unbiased N:M Sparsity for the Neural Gradients
par: Chmiel, Brian, et autres
Publié: (2022)

Block Sparse Flash Attention
par: Ohayon, Daniel, et autres
Publié: (2025)

Tensor-Parallelism with Partially Synchronized Activations
par: Lamprecht, Itay, et autres
Publié: (2025)

PLUMAGE: Probabilistic Low rank Unbiased Min Variance Gradient Estimator for Efficient Large Model Training
par: Haroush, Matan, et autres
Publié: (2025)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
par: Huang, Wei, et autres
Publié: (2025)

An Analytical Model for Overparameterized Learning Under Class Imbalance
par: Mor, Eliav, et autres
Publié: (2025)

Are Time Series Foundation Models Susceptible to Catastrophic Forgetting?
par: Karaouli, Nouha, et autres
Publié: (2025)

Predicting the Susceptibility of Examples to Catastrophic Forgetting
par: Hacohen, Guy, et autres
Publié: (2024)

Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System
par: Chen, Chen, et autres
Publié: (2024)

Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings
par: Nori, Milad Khademi, et autres
Publié: (2024)

Mitigating Catastrophic Forgetting in Language Transfer via Model Merging
par: Alexandrov, Anton, et autres
Publié: (2024)

Robust Policy Optimization to Prevent Catastrophic Forgetting
par: Sabbaghi, Mahdi, et autres
Publié: (2026)

Catastrophic Forgetting in Kolmogorov-Arnold Networks
par: Rahman, Mohammad Marufur, et autres
Publié: (2025)

Task Shift: From Classification to Regression in Overparameterized Linear Models
par: LaBonte, Tyler, et autres
Publié: (2025)

Avoid Catastrophic Forgetting with Rank-1 Fisher from Diffusion Models
par: Wang, Zekun, et autres
Publié: (2025)

Eidetic Learning: an Efficient and Provable Solution to Catastrophic Forgetting
par: Dronen, Nicholas, et autres
Publié: (2025)

Alleviating Catastrophic Forgetting in Facial Expression Recognition with Emotion-Centered Models
par: Laurensi, Israel A., et autres
Publié: (2024)

A representation-learning game for classes of prediction tasks
par: Uzan, Neria, et autres
Publié: (2024)

Exploration-Exploitation Tradeoff in Universal Lossy Compression
par: Weinberger, Nir, et autres
Publié: (2025)

Understanding Catastrophic Forgetting in Language Models via Implicit Inference
par: Kotha, Suhas, et autres
Publié: (2023)

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
par: Watts, Ishaan, et autres
Publié: (2026)

Sequencing to Mitigate Catastrophic Forgetting in Continual Learning
par: Moussa, Hesham G., et autres
Publié: (2025)

Federated Learning Under Temporal Drift -- Mitigating Catastrophic Forgetting via Experience Replay
par: Kokkula, Sahasra, et autres
Publié: (2026)

Estimation of Toeplitz Covariance Matrices using Overparameterized Gradient Descent
par: Busbib, Daniel, et autres
Publié: (2025)

Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning
par: Banayeeanzade, Amin, et autres
Publié: (2024)

Continual Learning and Catastrophic Forgetting
par: van de Ven, Gido M., et autres
Publié: (2024)

Quantum Maximum Likelihood Prediction via Hilbert Space Embeddings
par: Sreekumar, Sreejith, et autres
Publié: (2026)

Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
par: Elsayed, Mohamed, et autres
Publié: (2024)

Revisiting Catastrophic Forgetting in Continual Knowledge Graph Embedding
par: Pons, Gerard, et autres
Publié: (2026)

Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems
par: Banerjee, Sourasekhar, et autres
Publié: (2026)