When low-loss paths make a binary neuron trainable: detecting algorithmic transitions with the connected ensemble
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912863159844864 |
|---|---|
| author | Barbier, Damien |
| author_facet | Barbier, Damien |
| contents | We study the connected ensemble, a statistical-mechanics framework that characterizes the formation of low-loss paths in rugged landscapes. First introduced in a previous paper, this ensemble allows one to identify when a network can be trained on a simple task and which minima should be targeted during training. We apply this new framework to the symmetric binary perceptron model (SBP), and study how its typical {connected} minima behave. We show that {connected} minima exist only above a critical threshold $κ_{\rm connected}$, or equivalently below a critical constraint density $α_{\rm connected}$. This defines a parameter range in which training the network is easy, as local algorithms can efficiently access this connected manifold. We also highlight that these minima become increasingly robust and closer to one another as the task on which the network is trained becomes more difficult. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_23241 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | When low-loss paths make a binary neuron trainable: detecting algorithmic transitions with the connected ensemble Barbier, Damien Disordered Systems and Neural Networks Statistical Mechanics We study the connected ensemble, a statistical-mechanics framework that characterizes the formation of low-loss paths in rugged landscapes. First introduced in a previous paper, this ensemble allows one to identify when a network can be trained on a simple task and which minima should be targeted during training. We apply this new framework to the symmetric binary perceptron model (SBP), and study how its typical {connected} minima behave. We show that {connected} minima exist only above a critical threshold $κ_{\rm connected}$, or equivalently below a critical constraint density $α_{\rm connected}$. This defines a parameter range in which training the network is easy, as local algorithms can efficiently access this connected manifold. We also highlight that these minima become increasingly robust and closer to one another as the task on which the network is trained becomes more difficult. |
| title | When low-loss paths make a binary neuron trainable: detecting algorithmic transitions with the connected ensemble |
| topic | Disordered Systems and Neural Networks Statistical Mechanics |
| url | https://arxiv.org/abs/2601.23241 |