Scaling Graph Transformers: A Comparative Study of Sparse and Dense Attention
Fuente:
arXiv
Enregistré dans:
| Auteur principal: | |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911118921826304 |
|---|---|
| author | Dimitrov, Leon |
| author_facet | Dimitrov, Leon |
| contents | Graphs have become a central representation in machine learning for capturing relational and structured data across various domains. Traditional graph neural networks often struggle to capture long-range dependencies between nodes due to their local structure. Graph transformers overcome this by using attention mechanisms that allow nodes to exchange information globally. However, there are two types of attention in graph transformers: dense and sparse. In this paper, we compare these two attention mechanisms, analyze their trade-offs, and highlight when to use each. We also outline current challenges and problems in designing attention for graph transformers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17175 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Graph Transformers: A Comparative Study of Sparse and Dense Attention Dimitrov, Leon Machine Learning Artificial Intelligence Graphs have become a central representation in machine learning for capturing relational and structured data across various domains. Traditional graph neural networks often struggle to capture long-range dependencies between nodes due to their local structure. Graph transformers overcome this by using attention mechanisms that allow nodes to exchange information globally. However, there are two types of attention in graph transformers: dense and sparse. In this paper, we compare these two attention mechanisms, analyze their trade-offs, and highlight when to use each. We also outline current challenges and problems in designing attention for graph transformers. |
| title | Scaling Graph Transformers: A Comparative Study of Sparse and Dense Attention |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.17175 |