Residual connections provably mitigate oversmoothing in graph neural networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913634960015360 |
|---|---|
| author | Chen, Ziang Lin, Zhengjiang Chen, Shi Polyanskiy, Yury Rigollet, Philippe |
| author_facet | Chen, Ziang Lin, Zhengjiang Chen, Shi Polyanskiy, Yury Rigollet, Philippe |
| contents | Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00762 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Residual connections provably mitigate oversmoothing in graph neural networks Chen, Ziang Lin, Zhengjiang Chen, Shi Polyanskiy, Yury Rigollet, Philippe Machine Learning Dynamical Systems Probability Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments. |
| title | Residual connections provably mitigate oversmoothing in graph neural networks |
| topic | Machine Learning Dynamical Systems Probability |
| url | https://arxiv.org/abs/2501.00762 |