Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912107221483520 |
|---|---|
| author | Karella, Tomáš Harmanec, Adam Kotera, Jan Blažek, Jan Šroubek, Filip |
| author_facet | Karella, Tomáš Harmanec, Adam Kotera, Jan Blažek, Jan Šroubek, Filip |
| contents | CNNs exhibit inherent equivariance to image translation, leading to efficient parameter and data usage, faster learning, and improved robustness. The concept of translation equivariant networks has been successfully extended to rotation transformation using group convolution for discrete rotation groups and harmonic functions for the continuous rotation group encompassing $360^\circ$. We explore the compatibility of the SA mechanism with full rotation equivariance, in contrast to previous studies that focused on discrete rotation. We introduce the Harmformer, a harmonic transformer with a convolutional stem that achieves equivariance for both translation and continuous rotation. Accompanied by an end-to-end equivariance proof, the Harmformer not only outperforms previous equivariant transformers, but also demonstrates inherent stability under any continuous rotation, even without seeing rotated samples during training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03794 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance Karella, Tomáš Harmanec, Adam Kotera, Jan Blažek, Jan Šroubek, Filip Computer Vision and Pattern Recognition CNNs exhibit inherent equivariance to image translation, leading to efficient parameter and data usage, faster learning, and improved robustness. The concept of translation equivariant networks has been successfully extended to rotation transformation using group convolution for discrete rotation groups and harmonic functions for the continuous rotation group encompassing $360^\circ$. We explore the compatibility of the SA mechanism with full rotation equivariance, in contrast to previous studies that focused on discrete rotation. We introduce the Harmformer, a harmonic transformer with a convolutional stem that achieves equivariance for both translation and continuous rotation. Accompanied by an end-to-end equivariance proof, the Harmformer not only outperforms previous equivariant transformers, but also demonstrates inherent stability under any continuous rotation, even without seeing rotated samples during training. |
| title | Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.03794 |