Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915076973264896 |
|---|---|
| author | Frasca, Fabrizio Jogl, Fabian Eliasof, Moshe Ostrovsky, Matan Schönlieb, Carola-Bibiane Gärtner, Thomas Maron, Haggai |
| author_facet | Frasca, Fabrizio Jogl, Fabian Eliasof, Moshe Ostrovsky, Matan Schönlieb, Carola-Bibiane Gärtner, Thomas Maron, Haggai |
| contents | To develop a preliminary understanding towards Graph Foundation Models, we study the extent to which pretrained Graph Neural Networks can be applied across datasets, an effort requiring to be agnostic to dataset-specific features and their encodings. We build upon a purely structural pretraining approach and propose an extension to capture feature information while still being feature-agnostic. We evaluate pretrained models on downstream tasks for varying amounts of training samples and choices of pretraining datasets. Our preliminary results indicate that embeddings from pretrained models improve generalization only with enough downstream data points and in a degree which depends on the quantity and properties of pretraining data. Feature information can lead to improvements, but currently requires some similarities between pretraining and downstream feature spaces. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17609 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs Frasca, Fabrizio Jogl, Fabian Eliasof, Moshe Ostrovsky, Matan Schönlieb, Carola-Bibiane Gärtner, Thomas Maron, Haggai Machine Learning Neural and Evolutionary Computing To develop a preliminary understanding towards Graph Foundation Models, we study the extent to which pretrained Graph Neural Networks can be applied across datasets, an effort requiring to be agnostic to dataset-specific features and their encodings. We build upon a purely structural pretraining approach and propose an extension to capture feature information while still being feature-agnostic. We evaluate pretrained models on downstream tasks for varying amounts of training samples and choices of pretraining datasets. Our preliminary results indicate that embeddings from pretrained models improve generalization only with enough downstream data points and in a degree which depends on the quantity and properties of pretraining data. Feature information can lead to improvements, but currently requires some similarities between pretraining and downstream feature spaces. |
| title | Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs |
| topic | Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2412.17609 |