Spontaneous symmetry breaking and Goldstone modes for deep information propagation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913128854323200 |
|---|---|
| author | Iqbal, Nabil Keller, T. Anderson Song, Yue Miyato, Takeru Welling, Max |
| author_facet | Iqbal, Nabil Keller, T. Anderson Song, Yue Miyato, Takeru Welling, Max |
| contents | In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_14685 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Spontaneous symmetry breaking and Goldstone modes for deep information propagation Iqbal, Nabil Keller, T. Anderson Song, Yue Miyato, Takeru Welling, Max Machine Learning Statistical Mechanics Artificial Intelligence In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks. |
| title | Spontaneous symmetry breaking and Goldstone modes for deep information propagation |
| topic | Machine Learning Statistical Mechanics Artificial Intelligence |
| url | https://arxiv.org/abs/2605.14685 |