MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916634209288192 |
|---|---|
| author | Jiang, Haitian Liu, Renjie Huang, Zengfeng Wang, Yichuan Yan, Xiao Cai, Zhenkun Wang, Minjie Wipf, David |
| author_facet | Jiang, Haitian Liu, Renjie Huang, Zengfeng Wang, Yichuan Yan, Xiao Cai, Zhenkun Wang, Minjie Wipf, David |
| contents | Among the many variants of graph neural network (GNN) architectures capable of modeling data with cross-instance relations, an important subclass involves layers designed such that the forward pass iteratively reduces a graph-regularized energy function of interest. In this way, node embeddings produced at the output layer dually serve as both predictive features for solving downstream tasks (e.g., node classification) and energy function minimizers that inherit transparent, exploitable inductive biases and interpretability. However, scaling GNN architectures constructed in this way remains challenging, in part because the convergence of the forward pass may involve models with considerable depth. To tackle this limitation, we propose a sampling-based energy function and scalable GNN layers that iteratively reduce it, guided by convergence guarantees in certain settings. We also instantiate a full GNN architecture based on these designs, and the model achieves competitive accuracy and scalability when applied to the largest publicly-available node classification benchmark exceeding 1TB in size. Our source code is available at https://github.com/haitian-jiang/MuseGNN. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_12457 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy Jiang, Haitian Liu, Renjie Huang, Zengfeng Wang, Yichuan Yan, Xiao Cai, Zhenkun Wang, Minjie Wipf, David Machine Learning Among the many variants of graph neural network (GNN) architectures capable of modeling data with cross-instance relations, an important subclass involves layers designed such that the forward pass iteratively reduces a graph-regularized energy function of interest. In this way, node embeddings produced at the output layer dually serve as both predictive features for solving downstream tasks (e.g., node classification) and energy function minimizers that inherit transparent, exploitable inductive biases and interpretability. However, scaling GNN architectures constructed in this way remains challenging, in part because the convergence of the forward pass may involve models with considerable depth. To tackle this limitation, we propose a sampling-based energy function and scalable GNN layers that iteratively reduce it, guided by convergence guarantees in certain settings. We also instantiate a full GNN architecture based on these designs, and the model achieves competitive accuracy and scalability when applied to the largest publicly-available node classification benchmark exceeding 1TB in size. Our source code is available at https://github.com/haitian-jiang/MuseGNN. |
| title | MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2310.12457 |