Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.27290 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911721211297792 |
|---|---|
| author | Ng, Fisher Kutz, J. Nathan |
| author_facet | Ng, Fisher Kutz, J. Nathan |
| contents | Sequence models, and particularly Linear Recurrent Neural Networks (LRNNs) of the form $\mathbf{h}_{k+1} = \mathbf{W} \mathbf{h}_{k} + \mathbf{y}_k + \mathbf{b}$, are widely applicable in time-series analysis for dynamical systems, yet, as black-box algorithms, much is unknown about why they perform well. In this work, we leverage Takens' embedding theorem, which provides conditions under which partially observed time series organized into delay-coordinate vectors can faithfully represent the original system's dynamics, as a theoretical framework for explaining how and why sequence models preserve and reconstruct dynamical systems. For LRNNs, concatenating output states into delay-coordinate vectors gives rise to a ``delay" matrix $\mathbb{M}_{n,m}\in \mathbb{C}^{(nm) \times (n+1)m}$: a block matrix consisting of identity matrices $\mathbf{I} \in \mathbb{R}^{m \times m}$ repeated $n$ times along the main diagonal and weight matrices $\mathbf{W} \in \mathbb{C}^{m \times m}$ featured $n$ times along the super-diagonal. $\mathbb{M}_{n,m}$ relates the delay-coordinates of the input time series to those of the LRNN output states, and, for $\mathbb{M}_{n,m}$ to be an embedding, it must be full row-rank. We provide explicit conditions for $\mathbb{M}_{n,m}$ to be full row-rank and prove the condition number of $\mathbb{M}_{n,m}$ and determinant of $\mathbb{M}_{n,m} \mathbb{M}_{n,m}^*$--measures of embedding stability--are bounded independent of $n$, at least for certain ranges of $\mathbf{W}$'s singular values: namely, when $σ_{\max}(\mathbf{W}) \le 1$. This result explains why the spectrum of $\mathbf{W}$ for trained LRNNs tends to converge to within the unit circle. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_27290 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Linear Recurrent Neural Networks as Time-Delay Embeddings Ng, Fisher Kutz, J. Nathan Dynamical Systems 15A12, 15A18, 37C05, 39A05 Sequence models, and particularly Linear Recurrent Neural Networks (LRNNs) of the form $\mathbf{h}_{k+1} = \mathbf{W} \mathbf{h}_{k} + \mathbf{y}_k + \mathbf{b}$, are widely applicable in time-series analysis for dynamical systems, yet, as black-box algorithms, much is unknown about why they perform well. In this work, we leverage Takens' embedding theorem, which provides conditions under which partially observed time series organized into delay-coordinate vectors can faithfully represent the original system's dynamics, as a theoretical framework for explaining how and why sequence models preserve and reconstruct dynamical systems. For LRNNs, concatenating output states into delay-coordinate vectors gives rise to a ``delay" matrix $\mathbb{M}_{n,m}\in \mathbb{C}^{(nm) \times (n+1)m}$: a block matrix consisting of identity matrices $\mathbf{I} \in \mathbb{R}^{m \times m}$ repeated $n$ times along the main diagonal and weight matrices $\mathbf{W} \in \mathbb{C}^{m \times m}$ featured $n$ times along the super-diagonal. $\mathbb{M}_{n,m}$ relates the delay-coordinates of the input time series to those of the LRNN output states, and, for $\mathbb{M}_{n,m}$ to be an embedding, it must be full row-rank. We provide explicit conditions for $\mathbb{M}_{n,m}$ to be full row-rank and prove the condition number of $\mathbb{M}_{n,m}$ and determinant of $\mathbb{M}_{n,m} \mathbb{M}_{n,m}^*$--measures of embedding stability--are bounded independent of $n$, at least for certain ranges of $\mathbf{W}$'s singular values: namely, when $σ_{\max}(\mathbf{W}) \le 1$. This result explains why the spectrum of $\mathbf{W}$ for trained LRNNs tends to converge to within the unit circle. |
| title | Linear Recurrent Neural Networks as Time-Delay Embeddings |
| topic | Dynamical Systems 15A12, 15A18, 37C05, 39A05 |
| url | https://arxiv.org/abs/2605.27290 |