Daydreaming Hopfield Networks and their surprising effectiveness on correlated data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912404417282048 |
|---|---|
| author | Serricchio, Ludovica Bocchi, Dario Chilin, Claudio Marino, Raffaele Negri, Matteo Cammarota, Chiara Ricci-Tersenghi, Federico |
| author_facet | Serricchio, Ludovica Bocchi, Dario Chilin, Claudio Marino, Raffaele Negri, Matteo Cammarota, Chiara Ricci-Tersenghi, Federico |
| contents | To improve the storage capacity of the Hopfield model, we develop a version of the dreaming algorithm that perpetually reinforces the patterns to be stored (as in the Hebb rule), and erases the spurious memories (as in dreaming algorithms). For this reason, we called it Daydreaming. Daydreaming is not destructive and it converges asymptotically to stationary retrieval maps. When trained on random uncorrelated examples, the model shows optimal performance in terms of the size of the basins of attraction of stored examples and the quality of reconstruction. We also train the Daydreaming algorithm on correlated data obtained via the random-features model and argue that it spontaneously exploits the correlations thus increasing even further the storage capacity and the size of the basins of attraction. Moreover, the Daydreaming algorithm is also able to stabilize the features hidden in the data. Finally, we test Daydreaming on the MNIST dataset and show that it still works surprisingly well, producing attractors that are close to unseen examples and class prototypes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_08777 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Daydreaming Hopfield Networks and their surprising effectiveness on correlated data Serricchio, Ludovica Bocchi, Dario Chilin, Claudio Marino, Raffaele Negri, Matteo Cammarota, Chiara Ricci-Tersenghi, Federico Disordered Systems and Neural Networks To improve the storage capacity of the Hopfield model, we develop a version of the dreaming algorithm that perpetually reinforces the patterns to be stored (as in the Hebb rule), and erases the spurious memories (as in dreaming algorithms). For this reason, we called it Daydreaming. Daydreaming is not destructive and it converges asymptotically to stationary retrieval maps. When trained on random uncorrelated examples, the model shows optimal performance in terms of the size of the basins of attraction of stored examples and the quality of reconstruction. We also train the Daydreaming algorithm on correlated data obtained via the random-features model and argue that it spontaneously exploits the correlations thus increasing even further the storage capacity and the size of the basins of attraction. Moreover, the Daydreaming algorithm is also able to stabilize the features hidden in the data. Finally, we test Daydreaming on the MNIST dataset and show that it still works surprisingly well, producing attractors that are close to unseen examples and class prototypes. |
| title | Daydreaming Hopfield Networks and their surprising effectiveness on correlated data |
| topic | Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2405.08777 |