Ensembles provably learn equivariance through data augmentation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909969086939136 |
|---|---|
| author | Nordenfors, Oskar Flinth, Axel |
| author_facet | Nordenfors, Oskar Flinth, Axel |
| contents | Recently, it was proved that group equivariance emerges in ensembles of neural networks as the result of full augmentation in the limit of infinitely wide neural networks (neural tangent kernel limit). In this paper, we extend this result significantly. We provide a proof that this emergence does not depend on the neural tangent kernel limit at all. We also consider stochastic settings, and furthermore general architectures. For the latter, we provide a simple sufficient condition on the relation between the architecture and the action of the group for our results to hold. We validate our findings through simple numeric experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01452 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Ensembles provably learn equivariance through data augmentation Nordenfors, Oskar Flinth, Axel Machine Learning Numerical Analysis 68T07 (primary), 3799, 20C35 Recently, it was proved that group equivariance emerges in ensembles of neural networks as the result of full augmentation in the limit of infinitely wide neural networks (neural tangent kernel limit). In this paper, we extend this result significantly. We provide a proof that this emergence does not depend on the neural tangent kernel limit at all. We also consider stochastic settings, and furthermore general architectures. For the latter, we provide a simple sufficient condition on the relation between the architecture and the action of the group for our results to hold. We validate our findings through simple numeric experiments. |
| title | Ensembles provably learn equivariance through data augmentation |
| topic | Machine Learning Numerical Analysis 68T07 (primary), 3799, 20C35 |
| url | https://arxiv.org/abs/2410.01452 |