Space filling positionality and the Spiroformer
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916838701531136 |
|---|---|
| author | Maurin, M. Evangelista-Alvarado, M. Á. Suárez-Serrato, P. |
| author_facet | Maurin, M. Evangelista-Alvarado, M. Á. Suárez-Serrato, P. |
| contents | Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the $2$-sphere. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_08456 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Space filling positionality and the Spiroformer Maurin, M. Evangelista-Alvarado, M. Á. Suárez-Serrato, P. Machine Learning Artificial Intelligence Differential Geometry Dynamical Systems Symplectic Geometry Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the $2$-sphere. |
| title | Space filling positionality and the Spiroformer |
| topic | Machine Learning Artificial Intelligence Differential Geometry Dynamical Systems Symplectic Geometry |
| url | https://arxiv.org/abs/2507.08456 |