Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.07407 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912254553751552 |
|---|---|
| author | Franks, Billy Joe Eliasof, Moshe Cantürk, Semih Wolf, Guy Schönlieb, Carola-Bibiane Fellenz, Sophie Kloft, Marius |
| author_facet | Franks, Billy Joe Eliasof, Moshe Cantürk, Semih Wolf, Guy Schönlieb, Carola-Bibiane Fellenz, Sophie Kloft, Marius |
| contents | Recent advances in integrating positional and structural encodings (PSEs) into graph neural networks (GNNs) have significantly enhanced their performance across various graph learning tasks. However, the general applicability of these encodings and their potential to serve as foundational representations for graphs remain uncertain. This paper investigates the fine-tuning efficiency, scalability with sample size, and generalization capability of learnable PSEs across diverse graph datasets. Specifically, we evaluate their potential as universal pre-trained models that can be easily adapted to new tasks with minimal fine-tuning and limited data. Furthermore, we assess the expressivity of the learned representations, particularly, when used to augment downstream GNNs. We demonstrate through extensive benchmarking and empirical analysis that PSEs generally enhance downstream models. However, some datasets may require specific PSE-augmentations to achieve optimal performance. Nevertheless, our findings highlight their significant potential to become integral components of future graph foundation models. We provide new insights into the strengths and limitations of PSEs, contributing to the broader discourse on foundation models in graph learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07407 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Graph Foundation Models: A Study on the Generalization of Positional and Structural Encodings Franks, Billy Joe Eliasof, Moshe Cantürk, Semih Wolf, Guy Schönlieb, Carola-Bibiane Fellenz, Sophie Kloft, Marius Machine Learning Recent advances in integrating positional and structural encodings (PSEs) into graph neural networks (GNNs) have significantly enhanced their performance across various graph learning tasks. However, the general applicability of these encodings and their potential to serve as foundational representations for graphs remain uncertain. This paper investigates the fine-tuning efficiency, scalability with sample size, and generalization capability of learnable PSEs across diverse graph datasets. Specifically, we evaluate their potential as universal pre-trained models that can be easily adapted to new tasks with minimal fine-tuning and limited data. Furthermore, we assess the expressivity of the learned representations, particularly, when used to augment downstream GNNs. We demonstrate through extensive benchmarking and empirical analysis that PSEs generally enhance downstream models. However, some datasets may require specific PSE-augmentations to achieve optimal performance. Nevertheless, our findings highlight their significant potential to become integral components of future graph foundation models. We provide new insights into the strengths and limitations of PSEs, contributing to the broader discourse on foundation models in graph learning. |
| title | Towards Graph Foundation Models: A Study on the Generalization of Positional and Structural Encodings |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.07407 |