Graph Transformers: A Survey
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915687041073152 |
|---|---|
| author | Shehzad, Ahsan Xia, Feng Abid, Shagufta Peng, Ciyuan Yu, Shuo Zhang, Dongyu Verspoor, Karin |
| author_facet | Shehzad, Ahsan Xia, Feng Abid, Shagufta Peng, Ciyuan Yu, Shuo Zhang, Dongyu Verspoor, Karin |
| contents | Graph transformers are a recent advancement in machine learning, offering a new class of neural network models for graph-structured data. The synergy between transformers and graph learning demonstrates strong performance and versatility across various graph-related tasks. This survey provides an in-depth review of recent progress and challenges in graph transformer research. We begin with foundational concepts of graphs and transformers. We then explore design perspectives of graph transformers, focusing on how they integrate graph inductive biases and graph attention mechanisms into the transformer architecture. Furthermore, we propose a taxonomy classifying graph transformers based on depth, scalability, and pre-training strategies, summarizing key principles for effective development of graph transformer models. Beyond technical analysis, we discuss the applications of graph transformer models for node-level, edge-level, and graph-level tasks, exploring their potential in other application scenarios as well. Finally, we identify remaining challenges in the field, such as scalability and efficiency, generalization and robustness, interpretability and explainability, dynamic and complex graphs, as well as data quality and diversity, charting future directions for graph transformer research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_09777 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Graph Transformers: A Survey Shehzad, Ahsan Xia, Feng Abid, Shagufta Peng, Ciyuan Yu, Shuo Zhang, Dongyu Verspoor, Karin Machine Learning Artificial Intelligence 68T07, 68T05, 68U01 I.2.6 Graph transformers are a recent advancement in machine learning, offering a new class of neural network models for graph-structured data. The synergy between transformers and graph learning demonstrates strong performance and versatility across various graph-related tasks. This survey provides an in-depth review of recent progress and challenges in graph transformer research. We begin with foundational concepts of graphs and transformers. We then explore design perspectives of graph transformers, focusing on how they integrate graph inductive biases and graph attention mechanisms into the transformer architecture. Furthermore, we propose a taxonomy classifying graph transformers based on depth, scalability, and pre-training strategies, summarizing key principles for effective development of graph transformer models. Beyond technical analysis, we discuss the applications of graph transformer models for node-level, edge-level, and graph-level tasks, exploring their potential in other application scenarios as well. Finally, we identify remaining challenges in the field, such as scalability and efficiency, generalization and robustness, interpretability and explainability, dynamic and complex graphs, as well as data quality and diversity, charting future directions for graph transformer research. |
| title | Graph Transformers: A Survey |
| topic | Machine Learning Artificial Intelligence 68T07, 68T05, 68U01 I.2.6 |
| url | https://arxiv.org/abs/2407.09777 |