Cover Image

Large-Step Training Dynamics of a Two-Factor Linear Transformer Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Balasubramanian, Krishnakumar
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Dynamical Systems
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Identifying Equivalent Training Dynamics
by: Redman, William T., et al.
Published: (2023)

Bayesian Inference in Recurrent Explicit Duration Switching Linear Dynamical Systems
by: Słupiński, Mikołaj, et al.
Published: (2024)

Dynamical Systems Theory Behind a Hierarchical Reasoning Model
by: Es'kin, Vasiliy A., et al.
Published: (2026)

Position: Why a Dynamical Systems Perspective is Needed to Advance Time Series Modeling
by: Durstewitz, Daniel, et al.
Published: (2026)

Out-of-Domain Generalization in Dynamical Systems Reconstruction
by: Göring, Niclas, et al.
Published: (2024)

Transformers Handle Endogeneity in In-Context Linear Regression
by: Liang, Haodong, et al.
Published: (2024)

Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction
by: Hemmer, Christoph Jürgen, et al.
Published: (2024)

Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models
by: Balasubramanian, Krishnakumar
Published: (2026)

Hybrid Energy-Based Models for Physical AI: Provably Stable Identification of Port-Hamiltonian Dynamics
by: Betteti, Simone, et al.
Published: (2026)

True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics
by: Hemmer, Christoph Jürgen, et al.
Published: (2025)

KPFlow: An Operator Perspective on Dynamic Collapse Under Gradient Descent Training of Recurrent Networks
by: Hazelden, James, et al.
Published: (2025)

Manifold Trajectories in Next-Token Prediction: From Replicator Dynamics to Softmax Equilibrium
by: Lee-Jenkins, Christopher R.
Published: (2025)

A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Router
by: Kiselev, O. M.
Published: (2026)

Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction
by: Brenner, Manuel, et al.
Published: (2024)

The Recurrent Sticky Hierarchical Dirichlet Process Hidden Markov Model
by: Słupiński, Mikołaj, et al.
Published: (2024)

Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
by: Hoover, Benjamin, et al.
Published: (2023)

Kernel Dynamics under Path Entropy Maximization
by: Das, Jnaneshwar
Published: (2026)

A Hybrid Approach of Transfer Learning and Physics-Informed Modeling: Improving Dissolved Oxygen Concentration Prediction in an Industrial Wastewater Treatment Plant
by: Koksal, Ece S., et al.
Published: (2024)

A Quantitative Characterization of Forgetting in Post-Training
by: Balasubramanian, Krishnakumar, et al.
Published: (2026)

On the Limitations of Fractal Dimension as a Measure of Generalization
by: Tan, Charlie B., et al.
Published: (2024)

Analysis of Thompson Sampling for Controlling Unknown Linear Diffusion Processes
by: Faradonbeh, Mohamad Kazem Shirani, et al.
Published: (2022)

Confabulation dynamics in a reservoir computer: Filling in the gaps with untrained attractors
by: O'Hagan, Jack, et al.
Published: (2025)

The Asymptotic Behavior of Attention in Transformers
by: Abella, Álvaro Rodríguez, et al.
Published: (2024)

Horizon-Constrained Rashomon Sets for Chaotic Forecasting
by: Kale, Gauri, et al.
Published: (2026)

A Mathematical Conflict Framework for Contextual Data Modulation
by: Kartal, Hakan Emre
Published: (2026)

Inverse Depth Scaling From Most Layers Being Similar
by: Liu, Yizhou, et al.
Published: (2026)

Late Fusion Neural Operators for Extrapolation Across Parameter Space in Partial Differential Equations
by: van Tegelen, Eva, et al.
Published: (2026)

A-PINN: Auxiliary Physics-informed Neural Networks for Structural Vibration Analysis in Continuous Euler-Bernoulli Beam
by: Saini, Shivani, et al.
Published: (2025)

Language models as master equation solvers
by: Liu, Chuanbo, et al.
Published: (2023)

Detecting Invariant Manifolds in ReLU-Based RNNs
by: Eisenmann, Lukas, et al.
Published: (2025)

Decomposing heterogeneous dynamical systems with graph neural networks
by: Allier, Cédric, et al.
Published: (2024)

Improved deep learning of chaotic dynamical systems with multistep penalty losses
by: Chakraborty, Dibyajyoti, et al.
Published: (2024)

On the dimension of pullback attractors in recurrent neural networks
by: Fadera, Muhammed
Published: (2025)

Inverse Approximation Theory for Nonlinear Recurrent Neural Networks
by: Wang, Shida, et al.
Published: (2023)

Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networks
by: Gao, Mars Liyao, et al.
Published: (2025)

Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data
by: Brenner, Manuel, et al.
Published: (2024)

Planning Neural Dynamics with Lie Group Embedding through Supervised Projective Manifold Learning
by: Wang, Tianwei, et al.
Published: (2026)

LongSSM: On the Length Extension of State-space Models in Language Modelling
by: Wang, Shida
Published: (2024)

When is a System Discoverable from Data? Discovery Requires Chaos
by: Shumaylov, Zakhar, et al.
Published: (2025)

How to Tame Your LLM: Semantic Collapse in Continuous Systems
by: Wyss, C. M.
Published: (2025)