Spacetime $E(n)$-Transformer: Equivariant Attention for Spatio-temporal Graphs
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911984004366336 |
|---|---|
| author | Charles, Sergio G. |
| author_facet | Charles, Sergio G. |
| contents | We introduce an $E(n)$-equivariant Transformer architecture for spatio-temporal graph data. By imposing rotation, translation, and permutation equivariance inductive biases in both space and time, we show that the Spacetime $E(n)$-Transformer (SET) outperforms purely spatial and temporal models without symmetry-preserving properties. We benchmark SET against said models on the charged $N$-body problem, a simple physical system with complex dynamics. While existing spatio-temporal graph neural networks focus on sequential modeling, we empirically demonstrate that leveraging underlying domain symmetries yields considerable improvements for modeling dynamical systems on graphs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_06039 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Spacetime $E(n)$-Transformer: Equivariant Attention for Spatio-temporal Graphs Charles, Sergio G. Machine Learning Artificial Intelligence We introduce an $E(n)$-equivariant Transformer architecture for spatio-temporal graph data. By imposing rotation, translation, and permutation equivariance inductive biases in both space and time, we show that the Spacetime $E(n)$-Transformer (SET) outperforms purely spatial and temporal models without symmetry-preserving properties. We benchmark SET against said models on the charged $N$-body problem, a simple physical system with complex dynamics. While existing spatio-temporal graph neural networks focus on sequential modeling, we empirically demonstrate that leveraging underlying domain symmetries yields considerable improvements for modeling dynamical systems on graphs. |
| title | Spacetime $E(n)$-Transformer: Equivariant Attention for Spatio-temporal Graphs |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2408.06039 |