Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918294543400960 |
|---|---|
| author | Murray, Michael Chan, Tenzin Karhadker, Kedar Hillar, Christopher J. |
| author_facet | Murray, Michael Chan, Tenzin Karhadker, Kedar Hillar, Christopher J. |
| contents | Many learning problems involve symmetries, and while invariance can be built into neural architectures, it can also emerge implicitly when training on group-structured data. We study this phenomenon in classical Hopfield networks and show they can infer the full isomorphism class of a graph from a small random sample. Our results reveal that: (i) graph isomorphism classes can be represented within a three-dimensional invariant subspace, (ii) using gradient descent to minimize energy flow (MEF) has an implicit bias toward norm-efficient solutions, which underpins a polynomial sample complexity bound for learning isomorphism classes, and (iii) across multiple learning rules, parameters converge toward the invariant subspace as sample sizes grow. Together, these findings highlight a unifying mechanism for generalization in Hopfield networks: a bias toward norm efficiency in learning drives the emergence of approximate invariance under group-structured data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14338 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits Murray, Michael Chan, Tenzin Karhadker, Kedar Hillar, Christopher J. Machine Learning 68T07, 05C90 I.2.6; G.2.2 Many learning problems involve symmetries, and while invariance can be built into neural architectures, it can also emerge implicitly when training on group-structured data. We study this phenomenon in classical Hopfield networks and show they can infer the full isomorphism class of a graph from a small random sample. Our results reveal that: (i) graph isomorphism classes can be represented within a three-dimensional invariant subspace, (ii) using gradient descent to minimize energy flow (MEF) has an implicit bias toward norm-efficient solutions, which underpins a polynomial sample complexity bound for learning isomorphism classes, and (iii) across multiple learning rules, parameters converge toward the invariant subspace as sample sizes grow. Together, these findings highlight a unifying mechanism for generalization in Hopfield networks: a bias toward norm efficiency in learning drives the emergence of approximate invariance under group-structured data. |
| title | Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits |
| topic | Machine Learning 68T07, 05C90 I.2.6; G.2.2 |
| url | https://arxiv.org/abs/2512.14338 |