Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variants
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908645813387264 |
|---|---|
| author | Deng, Gangda Zhou, Hongkuan Kannan, Rajgopal Prasanna, Viktor |
| author_facet | Deng, Gangda Zhou, Hongkuan Kannan, Rajgopal Prasanna, Viktor |
| contents | Heterophilous graphs, where dissimilar nodes tend to connect, pose a challenge for graph neural networks (GNNs). Increasing the GNN depth can expand the scope (i.e., receptive field), potentially finding homophily from the higher-order neighborhoods. However, GNNs suffer from performance degradation as depth increases. Despite having better expressivity, state-of-the-art deeper GNNs achieve only marginal improvements compared to their shallow variants. Through theoretical and empirical analysis, we systematically demonstrate a shift in GNN generalization preferences across nodes with different homophily levels as depth increases. This creates a disparity in generalization patterns between GNN models with varying depth. Based on these findings, we propose to improve deeper GNN generalization while maintaining high expressivity by Mixture of scope experts at test (Moscat). Experimental results show that Moscat works flexibly with various GNNs across a wide range of datasets while significantly improving accuracy. Our code is available at (https://github.com/Hydrapse/moscat). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_06998 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variants Deng, Gangda Zhou, Hongkuan Kannan, Rajgopal Prasanna, Viktor Machine Learning Social and Information Networks Heterophilous graphs, where dissimilar nodes tend to connect, pose a challenge for graph neural networks (GNNs). Increasing the GNN depth can expand the scope (i.e., receptive field), potentially finding homophily from the higher-order neighborhoods. However, GNNs suffer from performance degradation as depth increases. Despite having better expressivity, state-of-the-art deeper GNNs achieve only marginal improvements compared to their shallow variants. Through theoretical and empirical analysis, we systematically demonstrate a shift in GNN generalization preferences across nodes with different homophily levels as depth increases. This creates a disparity in generalization patterns between GNN models with varying depth. Based on these findings, we propose to improve deeper GNN generalization while maintaining high expressivity by Mixture of scope experts at test (Moscat). Experimental results show that Moscat works flexibly with various GNNs across a wide range of datasets while significantly improving accuracy. Our code is available at (https://github.com/Hydrapse/moscat). |
| title | Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variants |
| topic | Machine Learning Social and Information Networks |
| url | https://arxiv.org/abs/2409.06998 |