Imagen de Portada

Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Koubbi, Hugo, Hernandez, Louis, Boussard, Matthieu
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Dynamical Systems
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Dynamic metastability in the self-attention model
por: Geshkovski, Borjan, et al.
Publicado: (2024)

The Mean-Field Dynamics of Transformers
por: Rigollet, Philippe
Publicado: (2025)

Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics
por: Smart, Matthew, et al.
Publicado: (2026)

Multistability of Self-Attention Dynamics in Transformers
por: Altafini, Claudio
Publicado: (2025)

Quantitative Clustering in Mean-Field Transformer Models
por: Chen, Shi, et al.
Publicado: (2025)

LoRA Learns Less and Forgets Less
por: Biderman, Dan, et al.
Publicado: (2024)

Convergent Stochastic Training of Attention and Understanding LoRA
por: Sun, Zhengkai, et al.
Publicado: (2026)

Signature Maximum Mean Discrepancy Two-Sample Statistical Tests
por: Alden, Andrew, et al.
Publicado: (2025)

Enhancing Neural Training via a Correlated Dynamics Model
por: Brokman, Jonathan, et al.
Publicado: (2023)

Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction
por: Wilson, Dan, et al.
Publicado: (2026)

From Mean-Field Limits to Semiclassical Concentration: Global Convergence of the Canonical Evolutionary Strategy
por: Neto, Matías, et al.
Publicado: (2026)

Backpropagation on Dynamical Networks
por: Tan, Eugene, et al.
Publicado: (2022)

LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
por: Li, Wenbing, et al.
Publicado: (2025)

From Sparse Sensors to Continuous Fields: STRIDE for Spatiotemporal Reconstruction
por: Tong, Yanjie, et al.
Publicado: (2026)

Dynamics-Informed Deep Learning for Predicting Extreme Events
por: Katsidoniotaki, Eirini, et al.
Publicado: (2026)

Multitask Learning with Stochastic Interpolants
por: Negrel, Hugo, et al.
Publicado: (2025)

Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics
por: Brenner, Manuel, et al.
Publicado: (2022)

Run LoRA Run: Faster and Lighter LoRA Implementations
por: Cherniuk, Daria, et al.
Publicado: (2023)

Merge before Forget: A Single LoRA Continual Learning via Continual Merging
por: Qiao, Fuli, et al.
Publicado: (2025)

Analysis of tidal flows through the Strait of Gibraltar using Dynamic Mode Decomposition
por: Dias, Sathsara, et al.
Publicado: (2023)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
por: Lee, Seungeon, et al.
Publicado: (2025)

Latent Diffeomorphic Dynamic Mode Decomposition
por: Diepeveen, Willem, et al.
Publicado: (2025)

Autoencoding Dynamics: Topological Limitations and Capabilities
por: Kvalheim, Matthew D., et al.
Publicado: (2025)

Deep Learning of the Evolution Operator Enables Forecasting of Out-of-Training Dynamics in Chaotic Systems
por: Shokar, Ira J. S., et al.
Publicado: (2025)

PINN-Obs: Physics-Informed Neural Network-Based Observer for Nonlinear Dynamical Systems
por: Farkane, Ayoub, et al.
Publicado: (2025)

CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting
por: Chen, Chuanqi, et al.
Publicado: (2023)

Likelihood-Free Inference and Hierarchical Data Assimilation for Geological Carbon Storage
por: Teng, Wenchao, et al.
Publicado: (2024)

Learning Deep Dissipative Dynamics
por: Okamoto, Yuji, et al.
Publicado: (2024)

Neural Context Flows for Meta-Learning of Dynamical Systems
por: Nzoyem, Roussel Desmond, et al.
Publicado: (2024)

Continuous-Time Homeostatic Dynamics for Reentrant Inference Models
por: Chae, Byung Gyu
Publicado: (2025)

Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
por: Seifner, Patrick, et al.
Publicado: (2024)

Dynamical systems and complex networks: A Koopman operator perspective
por: Klus, Stefan, et al.
Publicado: (2024)

Neural Ordinary Differential Equations for Modeling Socio-Economic Dynamics
por: Samota, Sandeep Kumar, et al.
Publicado: (2026)

Generalized Quadratic Embeddings for Nonlinear Dynamics using Deep Learning
por: Goyal, Pawan, et al.
Publicado: (2022)

Tipping Point Forecasting in Non-Stationary Dynamics on Function Spaces
por: Liu-Schiaffini, Miguel, et al.
Publicado: (2023)

Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems
por: Dang, Gang, et al.
Publicado: (2024)

DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems
por: Schiff, Yair, et al.
Publicado: (2024)

Embedded Variational Neural Stochastic Differential Equations for Learning Heterogeneous Dynamics
por: Samota, Sandeep Kumar, et al.
Publicado: (2026)

Flowing Through Layers: A Continuous Dynamical Systems Perspective on Transformers
por: Fein-Ashley, Jacob
Publicado: (2025)

Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems
por: Turri, Giacomo, et al.
Publicado: (2025)