Memory Capacity of Nonlinear Recurrent Networks: Is it Informative?
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912564763426816 |
|---|---|
| author | Ballarin, Giovanni Grigoryeva, Lyudmila Ortega, Juan-Pablo |
| author_facet | Ballarin, Giovanni Grigoryeva, Lyudmila Ortega, Juan-Pablo |
| contents | The total memory capacity (MC) of linear recurrent neural networks (RNNs) has been proven to be equal to the rank of the corresponding Kalman controllability matrix, and it is almost surely maximal for connectivity and input weight matrices drawn from regular distributions. This fact questions the usefulness of this metric in distinguishing the performance of linear RNNs in the processing of stochastic signals. This work shows that the MC of random nonlinear RNNs yields arbitrary values within established upper and lower bounds depending exclusively on the scale of the input process. This confirms that the existing definition of MC in linear and nonlinear cases has no practical value. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_04832 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Memory Capacity of Nonlinear Recurrent Networks: Is it Informative? Ballarin, Giovanni Grigoryeva, Lyudmila Ortega, Juan-Pablo Machine Learning The total memory capacity (MC) of linear recurrent neural networks (RNNs) has been proven to be equal to the rank of the corresponding Kalman controllability matrix, and it is almost surely maximal for connectivity and input weight matrices drawn from regular distributions. This fact questions the usefulness of this metric in distinguishing the performance of linear RNNs in the processing of stochastic signals. This work shows that the MC of random nonlinear RNNs yields arbitrary values within established upper and lower bounds depending exclusively on the scale of the input process. This confirms that the existing definition of MC in linear and nonlinear cases has no practical value. |
| title | Memory Capacity of Nonlinear Recurrent Networks: Is it Informative? |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.04832 |