Random Features Hopfield Networks generalize retrieval to previously unseen examples
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913420792561664 |
|---|---|
| author | Kalaj, Silvio Lauditi, Clarissa Perugini, Gabriele Lucibello, Carlo Malatesta, Enrico M. Negri, Matteo |
| author_facet | Kalaj, Silvio Lauditi, Clarissa Perugini, Gabriele Lucibello, Carlo Malatesta, Enrico M. Negri, Matteo |
| contents | It has been recently shown that a learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors corresponding to such features appear in the model. In this work we reveal that the network also develops attractors corresponding to previously unseen examples generated with the same set of features. We explain this surprising behaviour in terms of spurious states of the learned features: we argue that, increasing the number of stored examples beyond the learning transition, the model also learns to mix the features to represent both stored and previously unseen examples. We support this claim with the computation of the phase diagram of the model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_05658 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Random Features Hopfield Networks generalize retrieval to previously unseen examples Kalaj, Silvio Lauditi, Clarissa Perugini, Gabriele Lucibello, Carlo Malatesta, Enrico M. Negri, Matteo Disordered Systems and Neural Networks Machine Learning Neural and Evolutionary Computing It has been recently shown that a learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors corresponding to such features appear in the model. In this work we reveal that the network also develops attractors corresponding to previously unseen examples generated with the same set of features. We explain this surprising behaviour in terms of spurious states of the learned features: we argue that, increasing the number of stored examples beyond the learning transition, the model also learns to mix the features to represent both stored and previously unseen examples. We support this claim with the computation of the phase diagram of the model. |
| title | Random Features Hopfield Networks generalize retrieval to previously unseen examples |
| topic | Disordered Systems and Neural Networks Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2407.05658 |