Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912987042807808 |
|---|---|
| author | D'Amico, Francesco Bocchi, Dario Del Bono, Luca Maria Rossi, Saverio Negri, Matteo |
| author_facet | D'Amico, Francesco Bocchi, Dario Del Bono, Luca Maria Rossi, Saverio Negri, Matteo |
| contents | Energy-based probabilistic models learned by maximizing the likelihood of the data are limited by the intractability of the partition function. A widely used workaround is to maximize the pseudo-likelihood, which replaces the global normalization with tractable local normalizations. Here we show that, in the zero-temperature limit, a network trained to maximize pseudo-likelihood naturally implements an associative memory: if the training set is small, patterns become fixed-point attractors whose basins of attraction exceed those of any classical Hopfield rule. We explain quantitatively this effect on uncorrelated random patterns. Moreover, we show that, for different structured datasets coming from computer science (random feature model, MNIST), physics (spin glasses) and biology (proteins), as the number of training examples increases the learned network goes beyond memorization, developing meaningful attractors with non-trivial correlations with test examples, thus showing the ability to generalize. Our results therefore reveal pseudo-likelihood works both as an efficient inference tool and as a principled mechanism for memory and generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings D'Amico, Francesco Bocchi, Dario Del Bono, Luca Maria Rossi, Saverio Negri, Matteo Statistical Mechanics Disordered Systems and Neural Networks Machine Learning Energy-based probabilistic models learned by maximizing the likelihood of the data are limited by the intractability of the partition function. A widely used workaround is to maximize the pseudo-likelihood, which replaces the global normalization with tractable local normalizations. Here we show that, in the zero-temperature limit, a network trained to maximize pseudo-likelihood naturally implements an associative memory: if the training set is small, patterns become fixed-point attractors whose basins of attraction exceed those of any classical Hopfield rule. We explain quantitatively this effect on uncorrelated random patterns. Moreover, we show that, for different structured datasets coming from computer science (random feature model, MNIST), physics (spin glasses) and biology (proteins), as the number of training examples increases the learned network goes beyond memorization, developing meaningful attractors with non-trivial correlations with test examples, thus showing the ability to generalize. Our results therefore reveal pseudo-likelihood works both as an efficient inference tool and as a principled mechanism for memory and generalization. |
| title | Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings |
| topic | Statistical Mechanics Disordered Systems and Neural Networks Machine Learning |
| url | https://arxiv.org/abs/2507.05147 |