Graph Neural Networks Need Cluster-Normalize-Activate Modules
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929616856285184 |
|---|---|
| author | Skryagin, Arseny Divo, Felix Ali, Mohammad Amin Dhami, Devendra Singh Kersting, Kristian |
| author_facet | Skryagin, Arseny Divo, Felix Ali, Mohammad Amin Dhami, Devendra Singh Kersting, Kristian |
| contents | Graph Neural Networks (GNNs) are non-Euclidean deep learning models for graph-structured data. Despite their successful and diverse applications, oversmoothing prohibits deep architectures due to node features converging to a single fixed point. This severely limits their potential to solve complex tasks. To counteract this tendency, we propose a plug-and-play module consisting of three steps: Cluster-Normalize-Activate (CNA). By applying CNA modules, GNNs search and form super nodes in each layer, which are normalized and activated individually. We demonstrate in node classification and property prediction tasks that CNA significantly improves the accuracy over the state-of-the-art. Particularly, CNA reaches 94.18% and 95.75% accuracy on Cora and CiteSeer, respectively. It further benefits GNNs in regression tasks as well, reducing the mean squared error compared to all baselines. At the same time, GNNs with CNA require substantially fewer learnable parameters than competing architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04064 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Graph Neural Networks Need Cluster-Normalize-Activate Modules Skryagin, Arseny Divo, Felix Ali, Mohammad Amin Dhami, Devendra Singh Kersting, Kristian Machine Learning Artificial Intelligence 68T07 I.2.0 Graph Neural Networks (GNNs) are non-Euclidean deep learning models for graph-structured data. Despite their successful and diverse applications, oversmoothing prohibits deep architectures due to node features converging to a single fixed point. This severely limits their potential to solve complex tasks. To counteract this tendency, we propose a plug-and-play module consisting of three steps: Cluster-Normalize-Activate (CNA). By applying CNA modules, GNNs search and form super nodes in each layer, which are normalized and activated individually. We demonstrate in node classification and property prediction tasks that CNA significantly improves the accuracy over the state-of-the-art. Particularly, CNA reaches 94.18% and 95.75% accuracy on Cora and CiteSeer, respectively. It further benefits GNNs in regression tasks as well, reducing the mean squared error compared to all baselines. At the same time, GNNs with CNA require substantially fewer learnable parameters than competing architectures. |
| title | Graph Neural Networks Need Cluster-Normalize-Activate Modules |
| topic | Machine Learning Artificial Intelligence 68T07 I.2.0 |
| url | https://arxiv.org/abs/2412.04064 |