Dynamical Properties of Dense Associative Memory
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909872458563584 |
|---|---|
| author | Mimura, Kazushi Takeuchi, Jun'ichi Sumikawa, Yuto Kabashima, Yoshiyuki Coolen, Anthony C. C. |
| author_facet | Mimura, Kazushi Takeuchi, Jun'ichi Sumikawa, Yuto Kabashima, Yoshiyuki Coolen, Anthony C. C. |
| contents | Dense associative memory, a fundamental instance of modern Hopfield networks, can store a large number of memory patterns as equilibrium states of recurrent networks. While the stationary-state storage capacity has been investigated, its dynamical properties have not yet been discussed. In this paper, we analyze the dynamics using an exact approach based on generating functional analysis. We show results on convergence properties of memory retrieval, such as the convergence time and the size of the attraction basins. Our analysis enables a quantitative evaluation of the convergence time and the storage capacity of dense associative memory, which is useful for model design. Unlike the traditional Hopfield model, the retrieval of a pattern does not act as additional noise to itself, suggesting that the structure of modern networks makes recall more robust. Furthermore, the methodology addressed here can be applied to other energy-based models, and thus has the potential to contribute to the design of future architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00851 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamical Properties of Dense Associative Memory Mimura, Kazushi Takeuchi, Jun'ichi Sumikawa, Yuto Kabashima, Yoshiyuki Coolen, Anthony C. C. Disordered Systems and Neural Networks Dense associative memory, a fundamental instance of modern Hopfield networks, can store a large number of memory patterns as equilibrium states of recurrent networks. While the stationary-state storage capacity has been investigated, its dynamical properties have not yet been discussed. In this paper, we analyze the dynamics using an exact approach based on generating functional analysis. We show results on convergence properties of memory retrieval, such as the convergence time and the size of the attraction basins. Our analysis enables a quantitative evaluation of the convergence time and the storage capacity of dense associative memory, which is useful for model design. Unlike the traditional Hopfield model, the retrieval of a pattern does not act as additional noise to itself, suggesting that the structure of modern networks makes recall more robust. Furthermore, the methodology addressed here can be applied to other energy-based models, and thus has the potential to contribute to the design of future architectures. |
| title | Dynamical Properties of Dense Associative Memory |
| topic | Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2506.00851 |