Energy Guided smoothness to improve Robustness in Graph Classification
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915776454197248 |
|---|---|
| author | Wani, Farooq Ahmad Bucarelli, Maria Sofia Di Francesco, Andrea Giuseppe Pryymak, Oleksandr Silvestri, Fabrizio |
| author_facet | Wani, Farooq Ahmad Bucarelli, Maria Sofia Di Francesco, Andrea Giuseppe Pryymak, Oleksandr Silvestri, Fabrizio |
| contents | Graph Neural Networks (GNNs) are powerful at solving graph classification tasks, yet applied problems often contain noisy labels. In this work, we study GNN robustness to label noise, demonstrate GNN failure modes when models struggle to generalise on low-order graphs, low label coverage, or when a model is over-parameterized. We establish both empirical and theoretical links between GNN robustness and the reduction of the total Dirichlet Energy of learned node representations, which encapsulates the hypothesized GNN smoothness inductive bias. Finally, we introduce two training strategies to enhance GNN robustness: (1) by incorporating a novel inductive bias in the weight matrices through the removal of negative eigenvalues, connected to Dirichlet Energy minimization; (2) by extending to GNNs a loss penalty that promotes learned smoothness. Importantly, neither approach negatively impacts performance in noise-free settings, supporting our hypothesis that the source of GNNs robustness is their smoothness inductive bias. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_08419 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Energy Guided smoothness to improve Robustness in Graph Classification Wani, Farooq Ahmad Bucarelli, Maria Sofia Di Francesco, Andrea Giuseppe Pryymak, Oleksandr Silvestri, Fabrizio Machine Learning Graph Neural Networks (GNNs) are powerful at solving graph classification tasks, yet applied problems often contain noisy labels. In this work, we study GNN robustness to label noise, demonstrate GNN failure modes when models struggle to generalise on low-order graphs, low label coverage, or when a model is over-parameterized. We establish both empirical and theoretical links between GNN robustness and the reduction of the total Dirichlet Energy of learned node representations, which encapsulates the hypothesized GNN smoothness inductive bias. Finally, we introduce two training strategies to enhance GNN robustness: (1) by incorporating a novel inductive bias in the weight matrices through the removal of negative eigenvalues, connected to Dirichlet Energy minimization; (2) by extending to GNNs a loss penalty that promotes learned smoothness. Importantly, neither approach negatively impacts performance in noise-free settings, supporting our hypothesis that the source of GNNs robustness is their smoothness inductive bias. |
| title | Energy Guided smoothness to improve Robustness in Graph Classification |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.08419 |