Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2501.19089 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916741232197632 |
|---|---|
| author | Wang, Keqin Yang, Yulong Saha, Ishan Allen-Blanchette, Christine |
| author_facet | Wang, Keqin Yang, Yulong Saha, Ishan Allen-Blanchette, Christine |
| contents | While graph neural networks (GNNs) have allowed researchers to successfully apply neural networks to non-Euclidean domains, deep GNNs often exhibit lower predictive performance than their shallow counterparts. This phenomena has been attributed in part to oversmoothing, the tendency of node representations to become increasingly similar with network depth. In this paper we introduce an analogy between oversmoothing in GNNs and consensus (i.e., perfect agreement) in the opinion dynamics literature. We show that the message passing algorithms of several GNN models are equivalent to linear opinion dynamics models which have been shown to converge to consensus for all inputs regardless of the graph structure. This new perspective on oversmoothing motivates the use of nonlinear opinion dynamics as an inductive bias in GNN models. In our Behavior-Inspired Message Passing (BIMP) GNN, we leverage the nonlinear opinion dynamics model which is more general than the linear opinion dynamics model, and can be designed to converge to dissensus for general inputs. Through extensive experiments we show that BIMP resists oversmoothing beyond 100 time steps and consistently outperforms existing architectures even when those architectures are amended with oversmoothing mitigation techniques. We also show that BIMP has several desirable properties including well behaved gradients and adaptability to homophilic and heterophilic datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_19089 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Resolving Oversmoothing with Opinion Dissensus Wang, Keqin Yang, Yulong Saha, Ishan Allen-Blanchette, Christine Machine Learning While graph neural networks (GNNs) have allowed researchers to successfully apply neural networks to non-Euclidean domains, deep GNNs often exhibit lower predictive performance than their shallow counterparts. This phenomena has been attributed in part to oversmoothing, the tendency of node representations to become increasingly similar with network depth. In this paper we introduce an analogy between oversmoothing in GNNs and consensus (i.e., perfect agreement) in the opinion dynamics literature. We show that the message passing algorithms of several GNN models are equivalent to linear opinion dynamics models which have been shown to converge to consensus for all inputs regardless of the graph structure. This new perspective on oversmoothing motivates the use of nonlinear opinion dynamics as an inductive bias in GNN models. In our Behavior-Inspired Message Passing (BIMP) GNN, we leverage the nonlinear opinion dynamics model which is more general than the linear opinion dynamics model, and can be designed to converge to dissensus for general inputs. Through extensive experiments we show that BIMP resists oversmoothing beyond 100 time steps and consistently outperforms existing architectures even when those architectures are amended with oversmoothing mitigation techniques. We also show that BIMP has several desirable properties including well behaved gradients and adaptability to homophilic and heterophilic datasets. |
| title | Resolving Oversmoothing with Opinion Dissensus |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.19089 |