Platonic Transformers: A Solid Choice For Equivariance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916995831693312 |
|---|---|
| author | Islam, Mohammad Mohaiminul Anand, Rishabh Wessels, David R. de Kruiff, Friso Kuipers, Thijs P. Ying, Rex Sánchez, Clara I. Vadgama, Sharvaree Bökman, Georg Bekkers, Erik J. |
| author_facet | Islam, Mohammad Mohaiminul Anand, Rishabh Wessels, David R. de Kruiff, Friso Kuipers, Thijs P. Ying, Rex Sánchez, Clara I. Vadgama, Sharvaree Bökman, Georg Bekkers, Erik J. |
| contents | While widespread, Transformers lack inductive biases for geometric symmetries common in science and computer vision. Existing equivariant methods often sacrifice the efficiency and flexibility that make Transformers so effective through complex, computationally intensive designs. We introduce the Platonic Transformer to resolve this trade-off. By defining attention relative to reference frames from the Platonic solid symmetry groups, our method induces a principled weight-sharing scheme. This enables combined equivariance to continuous translations and Platonic symmetries, while preserving the exact architecture and computational cost of a standard Transformer. Furthermore, we show that this attention is formally equivalent to a dynamic group convolution, which reveals that the model learns adaptive geometric filters and enables a highly scalable, linear-time convolutional variant. Across diverse benchmarks in computer vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25), the Platonic Transformer achieves competitive performance by leveraging these geometric constraints at no additional cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03511 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Platonic Transformers: A Solid Choice For Equivariance Islam, Mohammad Mohaiminul Anand, Rishabh Wessels, David R. de Kruiff, Friso Kuipers, Thijs P. Ying, Rex Sánchez, Clara I. Vadgama, Sharvaree Bökman, Georg Bekkers, Erik J. Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Image and Video Processing While widespread, Transformers lack inductive biases for geometric symmetries common in science and computer vision. Existing equivariant methods often sacrifice the efficiency and flexibility that make Transformers so effective through complex, computationally intensive designs. We introduce the Platonic Transformer to resolve this trade-off. By defining attention relative to reference frames from the Platonic solid symmetry groups, our method induces a principled weight-sharing scheme. This enables combined equivariance to continuous translations and Platonic symmetries, while preserving the exact architecture and computational cost of a standard Transformer. Furthermore, we show that this attention is formally equivalent to a dynamic group convolution, which reveals that the model learns adaptive geometric filters and enables a highly scalable, linear-time convolutional variant. Across diverse benchmarks in computer vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25), the Platonic Transformer achieves competitive performance by leveraging these geometric constraints at no additional cost. |
| title | Platonic Transformers: A Solid Choice For Equivariance |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Image and Video Processing |
| url | https://arxiv.org/abs/2510.03511 |