How does over-squashing affect the power of GNNs?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929241452445696 |
|---|---|
| author | Di Giovanni, Francesco Rusch, T. Konstantin Bronstein, Michael M. Deac, Andreea Lackenby, Marc Mishra, Siddhartha Veličković, Petar |
| author_facet | Di Giovanni, Francesco Rusch, T. Konstantin Bronstein, Michael M. Deac, Andreea Lackenby, Marc Mishra, Siddhartha Veličković, Petar |
| contents | Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_03589 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | How does over-squashing affect the power of GNNs? Di Giovanni, Francesco Rusch, T. Konstantin Bronstein, Michael M. Deac, Andreea Lackenby, Marc Mishra, Siddhartha Veličković, Petar Machine Learning Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies. |
| title | How does over-squashing affect the power of GNNs? |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2306.03589 |