Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.05368 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910679092428800 |
|---|---|
| author | Zhou, Jingbo Du, Yixuan Zhang, Ruqiong Xia, Jun Yu, Zhizhi Zang, Zelin Jin, Di Yang, Carl Zhang, Rui Li, Stan Z. |
| author_facet | Zhou, Jingbo Du, Yixuan Zhang, Ruqiong Xia, Jun Yu, Zhizhi Zang, Zelin Jin, Di Yang, Carl Zhang, Rui Li, Stan Z. |
| contents | Graph Neural Networks (GNNs), a type of neural network that can learn from graph-structured data through neighborhood information aggregation, have shown superior performance in various downstream tasks. However, as the number of layers increases, node representations become indistinguishable, which is known as over-smoothing. To address this issue, many residual methods have emerged. In this paper, we focus on the over-smoothing issue and related residual methods. Firstly, we revisit over-smoothing from the perspective of overlapping neighborhood subgraphs, and based on this, we explain how residual methods can alleviate over-smoothing by integrating multiple orders neighborhood subgraphs to avoid the indistinguishability of the single high-order neighborhood subgraphs. Additionally, we reveal the drawbacks of previous residual methods, such as the lack of node adaptability and severe loss of high-order neighborhood subgraph information, and propose a \textbf{Posterior-Sampling-based, Node-Adaptive Residual module (PSNR)}. We theoretically demonstrate that PSNR can alleviate the drawbacks of previous residual methods. Furthermore, extensive experiments verify the superiority of the PSNR module in fully observed node classification and missing feature scenarios. Our code is available at https://github.com/jingbo02/PSNR-GNN. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_05368 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptive Residual Module Zhou, Jingbo Du, Yixuan Zhang, Ruqiong Xia, Jun Yu, Zhizhi Zang, Zelin Jin, Di Yang, Carl Zhang, Rui Li, Stan Z. Machine Learning Artificial Intelligence Graph Neural Networks (GNNs), a type of neural network that can learn from graph-structured data through neighborhood information aggregation, have shown superior performance in various downstream tasks. However, as the number of layers increases, node representations become indistinguishable, which is known as over-smoothing. To address this issue, many residual methods have emerged. In this paper, we focus on the over-smoothing issue and related residual methods. Firstly, we revisit over-smoothing from the perspective of overlapping neighborhood subgraphs, and based on this, we explain how residual methods can alleviate over-smoothing by integrating multiple orders neighborhood subgraphs to avoid the indistinguishability of the single high-order neighborhood subgraphs. Additionally, we reveal the drawbacks of previous residual methods, such as the lack of node adaptability and severe loss of high-order neighborhood subgraph information, and propose a \textbf{Posterior-Sampling-based, Node-Adaptive Residual module (PSNR)}. We theoretically demonstrate that PSNR can alleviate the drawbacks of previous residual methods. Furthermore, extensive experiments verify the superiority of the PSNR module in fully observed node classification and missing feature scenarios. Our code is available at https://github.com/jingbo02/PSNR-GNN. |
| title | Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptive Residual Module |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2305.05368 |