How to escape atypical regions in the symmetric binary perceptron: a journey through connected-solutions states
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913769705177088 |
|---|---|
| author | Barbier, Damien |
| author_facet | Barbier, Damien |
| contents | We study the binary symmetric perceptron model, and in particular its atypical solutions. While the solution-space of this problem is dominated by isolated configurations, it is also solvable for a certain range of constraint density $α$ and threshold $κ$. We provide in this paper a statistical measure probing sequences of solutions, where two consecutive elements shares a strong overlap. After simplifications, we test its predictions by comparing it to Monte-Carlo simulations. We obtain good agreement and show that connected states with a Markovian correlation profile can fully decorrelate from their initialization only for $κ>κ_{\rm no-mem.\, state}$ ($κ_{\rm no-mem.\, state}\sim \sqrt{0.91\log(N)}$ for $α=0.5$ and $N$ being the dimension of the problem). For $κ<κ_{\rm no-mem.\, state}$, we show that decorrelated sequences still exist but have a non-trivial correlations profile. To study this regime we introduce an $Ansatz$ for the correlations that we label as the nested Markov chain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04479 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | How to escape atypical regions in the symmetric binary perceptron: a journey through connected-solutions states Barbier, Damien Disordered Systems and Neural Networks Statistical Mechanics Mathematical Physics We study the binary symmetric perceptron model, and in particular its atypical solutions. While the solution-space of this problem is dominated by isolated configurations, it is also solvable for a certain range of constraint density $α$ and threshold $κ$. We provide in this paper a statistical measure probing sequences of solutions, where two consecutive elements shares a strong overlap. After simplifications, we test its predictions by comparing it to Monte-Carlo simulations. We obtain good agreement and show that connected states with a Markovian correlation profile can fully decorrelate from their initialization only for $κ>κ_{\rm no-mem.\, state}$ ($κ_{\rm no-mem.\, state}\sim \sqrt{0.91\log(N)}$ for $α=0.5$ and $N$ being the dimension of the problem). For $κ<κ_{\rm no-mem.\, state}$, we show that decorrelated sequences still exist but have a non-trivial correlations profile. To study this regime we introduce an $Ansatz$ for the correlations that we label as the nested Markov chain. |
| title | How to escape atypical regions in the symmetric binary perceptron: a journey through connected-solutions states |
| topic | Disordered Systems and Neural Networks Statistical Mechanics Mathematical Physics |
| url | https://arxiv.org/abs/2408.04479 |