Dense Associative Memory Through the Lens of Random Features
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909373894230016 |
|---|---|
| author | Hoover, Benjamin Chau, Duen Horng Strobelt, Hendrik Ram, Parikshit Krotov, Dmitry |
| author_facet | Hoover, Benjamin Chau, Duen Horng Strobelt, Hendrik Ram, Parikshit Krotov, Dmitry |
| contents | Dense Associative Memories are high storage capacity variants of the Hopfield networks that are capable of storing a large number of memory patterns in the weights of the network of a given size. Their common formulations typically require storing each pattern in a separate set of synaptic weights, which leads to the increase of the number of synaptic weights when new patterns are introduced. In this work we propose an alternative formulation of this class of models using random features, commonly used in kernel methods. In this formulation the number of network's parameters remains fixed. At the same time, new memories can be added to the network by modifying existing weights. We show that this novel network closely approximates the energy function and dynamics of conventional Dense Associative Memories and shares their desirable computational properties. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_24153 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dense Associative Memory Through the Lens of Random Features Hoover, Benjamin Chau, Duen Horng Strobelt, Hendrik Ram, Parikshit Krotov, Dmitry Machine Learning Dense Associative Memories are high storage capacity variants of the Hopfield networks that are capable of storing a large number of memory patterns in the weights of the network of a given size. Their common formulations typically require storing each pattern in a separate set of synaptic weights, which leads to the increase of the number of synaptic weights when new patterns are introduced. In this work we propose an alternative formulation of this class of models using random features, commonly used in kernel methods. In this formulation the number of network's parameters remains fixed. At the same time, new memories can be added to the network by modifying existing weights. We show that this novel network closely approximates the energy function and dynamics of conventional Dense Associative Memories and shares their desirable computational properties. |
| title | Dense Associative Memory Through the Lens of Random Features |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.24153 |