The Symmetries of Three-Layer ReLU Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911694084636672 |
|---|---|
| author | Gegenfurtner, Johanna Marie Grillo, Moritz Montúfar, Guido |
| author_facet | Gegenfurtner, Johanna Marie Grillo, Moritz Montúfar, Guido |
| contents | We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck architectures. Our approach provides explicit semi-algebraic descriptions of these fibers and yields a polynomial time algorithm for deciding functional equivalence of two parameters. The symmetries include discrete and continuous transformations arising from layer composition, and depend on whether deeper layers hide or preserve geometric structure from preceding layers. Finally, we show that some of these symmetries induce local conservation laws along gradient flow, while others do not. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18319 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Symmetries of Three-Layer ReLU Networks Gegenfurtner, Johanna Marie Grillo, Moritz Montúfar, Guido Machine Learning Discrete Mathematics Algebraic Geometry Combinatorics We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck architectures. Our approach provides explicit semi-algebraic descriptions of these fibers and yields a polynomial time algorithm for deciding functional equivalence of two parameters. The symmetries include discrete and continuous transformations arising from layer composition, and depend on whether deeper layers hide or preserve geometric structure from preceding layers. Finally, we show that some of these symmetries induce local conservation laws along gradient flow, while others do not. |
| title | The Symmetries of Three-Layer ReLU Networks |
| topic | Machine Learning Discrete Mathematics Algebraic Geometry Combinatorics |
| url | https://arxiv.org/abs/2605.18319 |