Position: Graph Foundation Models are Already Here
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910464629276672 |
|---|---|
| author | Mao, Haitao Chen, Zhikai Tang, Wenzhuo Zhao, Jianan Ma, Yao Zhao, Tong Shah, Neil Galkin, Mikhail Tang, Jiliang |
| author_facet | Mao, Haitao Chen, Zhikai Tang, Wenzhuo Zhao, Jianan Ma, Yao Zhao, Tong Shah, Neil Galkin, Mikhail Tang, Jiliang |
| contents | Graph Foundation Models (GFMs) are emerging as a significant research topic in the graph domain, aiming to develop graph models trained on extensive and diverse data to enhance their applicability across various tasks and domains. Developing GFMs presents unique challenges over traditional Graph Neural Networks (GNNs), which are typically trained from scratch for specific tasks on particular datasets. The primary challenge in constructing GFMs lies in effectively leveraging vast and diverse graph data to achieve positive transfer. Drawing inspiration from existing foundation models in the CV and NLP domains, we propose a novel perspective for the GFM development by advocating for a ``graph vocabulary'', in which the basic transferable units underlying graphs encode the invariance on graphs. We ground the graph vocabulary construction from essential aspects including network analysis, expressiveness, and stability. Such a vocabulary perspective can potentially advance the future GFM design in line with the neural scaling laws. All relevant resources with GFM design can be found here. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02216 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Position: Graph Foundation Models are Already Here Mao, Haitao Chen, Zhikai Tang, Wenzhuo Zhao, Jianan Ma, Yao Zhao, Tong Shah, Neil Galkin, Mikhail Tang, Jiliang Machine Learning Graph Foundation Models (GFMs) are emerging as a significant research topic in the graph domain, aiming to develop graph models trained on extensive and diverse data to enhance their applicability across various tasks and domains. Developing GFMs presents unique challenges over traditional Graph Neural Networks (GNNs), which are typically trained from scratch for specific tasks on particular datasets. The primary challenge in constructing GFMs lies in effectively leveraging vast and diverse graph data to achieve positive transfer. Drawing inspiration from existing foundation models in the CV and NLP domains, we propose a novel perspective for the GFM development by advocating for a ``graph vocabulary'', in which the basic transferable units underlying graphs encode the invariance on graphs. We ground the graph vocabulary construction from essential aspects including network analysis, expressiveness, and stability. Such a vocabulary perspective can potentially advance the future GFM design in line with the neural scaling laws. All relevant resources with GFM design can be found here. |
| title | Position: Graph Foundation Models are Already Here |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.02216 |