On the Utilization of Unique Node Identifiers in Graph Neural Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910695032881152 |
|---|---|
| author | Bechler-Speicher, Maya Eliasof, Moshe Schönlieb, Carola-Bibiane Gilad-Bachrach, Ran Globerson, Amir |
| author_facet | Bechler-Speicher, Maya Eliasof, Moshe Schönlieb, Carola-Bibiane Gilad-Bachrach, Ran Globerson, Amir |
| contents | Graph Neural Networks have inherent representational limitations due to their message-passing structure. Recent work has suggested that these limitations can be overcome by using unique node identifiers (UIDs). Here we argue that despite the advantages of UIDs, one of their disadvantages is that they lose the desirable property of permutation-equivariance. We thus propose to focus on UID models that are permutation-equivariant, and present theoretical arguments for their advantages. Motivated by this, we propose a method to regularize UID models towards permutation equivariance, via a contrastive loss. We empirically demonstrate that our approach improves generalization and extrapolation abilities while providing faster training convergence. On the recent BREC expressiveness benchmark, our proposed method achieves state-of-the-art performance compared to other random-based approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02271 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Utilization of Unique Node Identifiers in Graph Neural Networks Bechler-Speicher, Maya Eliasof, Moshe Schönlieb, Carola-Bibiane Gilad-Bachrach, Ran Globerson, Amir Machine Learning Artificial Intelligence Graph Neural Networks have inherent representational limitations due to their message-passing structure. Recent work has suggested that these limitations can be overcome by using unique node identifiers (UIDs). Here we argue that despite the advantages of UIDs, one of their disadvantages is that they lose the desirable property of permutation-equivariance. We thus propose to focus on UID models that are permutation-equivariant, and present theoretical arguments for their advantages. Motivated by this, we propose a method to regularize UID models towards permutation equivariance, via a contrastive loss. We empirically demonstrate that our approach improves generalization and extrapolation abilities while providing faster training convergence. On the recent BREC expressiveness benchmark, our proposed method achieves state-of-the-art performance compared to other random-based approaches. |
| title | On the Utilization of Unique Node Identifiers in Graph Neural Networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.02271 |