Differentiable Cluster Graph Neural Network
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914811595456512 |
|---|---|
| author | Dong, Yanfei Dupty, Mohammed Haroon Deng, Lambert Liu, Zhuanghua Goh, Yong Liang Lee, Wee Sun |
| author_facet | Dong, Yanfei Dupty, Mohammed Haroon Deng, Lambert Liu, Zhuanghua Goh, Yong Liang Lee, Wee Sun |
| contents | Graph Neural Networks often struggle with long-range information propagation and in the presence of heterophilous neighborhoods. We address both challenges with a unified framework that incorporates a clustering inductive bias into the message passing mechanism, using additional cluster-nodes. Central to our approach is the formulation of an optimal transport based implicit clustering objective function. However, the algorithm for solving the implicit objective function needs to be differentiable to enable end-to-end learning of the GNN. To facilitate this, we adopt an entropy regularized objective function and propose an iterative optimization process, alternating between solving for the cluster assignments and updating the node/cluster-node embeddings. Notably, our derived closed-form optimization steps are themselves simple yet elegant message passing steps operating seamlessly on a bipartite graph of nodes and cluster-nodes. Our clustering-based approach can effectively capture both local and global information, demonstrated by extensive experiments on both heterophilous and homophilous datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16185 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Differentiable Cluster Graph Neural Network Dong, Yanfei Dupty, Mohammed Haroon Deng, Lambert Liu, Zhuanghua Goh, Yong Liang Lee, Wee Sun Machine Learning Artificial Intelligence Graph Neural Networks often struggle with long-range information propagation and in the presence of heterophilous neighborhoods. We address both challenges with a unified framework that incorporates a clustering inductive bias into the message passing mechanism, using additional cluster-nodes. Central to our approach is the formulation of an optimal transport based implicit clustering objective function. However, the algorithm for solving the implicit objective function needs to be differentiable to enable end-to-end learning of the GNN. To facilitate this, we adopt an entropy regularized objective function and propose an iterative optimization process, alternating between solving for the cluster assignments and updating the node/cluster-node embeddings. Notably, our derived closed-form optimization steps are themselves simple yet elegant message passing steps operating seamlessly on a bipartite graph of nodes and cluster-nodes. Our clustering-based approach can effectively capture both local and global information, demonstrated by extensive experiments on both heterophilous and homophilous datasets. |
| title | Differentiable Cluster Graph Neural Network |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2405.16185 |