Sampling-based Distributed Training with Message Passing Neural Network
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929719658676224 |
|---|---|
| author | Kakka, Priyesh Nidhan, Sheel Ranade, Rishikesh Pathak, Jay MacArt, Jonathan F. |
| author_facet | Kakka, Priyesh Nidhan, Sheel Ranade, Rishikesh Pathak, Jay MacArt, Jonathan F. |
| contents | In this study, we introduce a domain-decomposition-based distributed training and inference approach for message-passing neural networks (MPNN). Our objective is to address the challenge of scaling edge-based graph neural networks as the number of nodes increases. Through our distributed training approach, coupled with Nyström-approximation sampling techniques, we present a scalable graph neural network, referred to as DS-MPNN (D and S standing for distributed and sampled, respectively), capable of scaling up to $O(10^5)$ nodes. We validate our sampling and distributed training approach on two cases: (a) a Darcy flow dataset and (b) steady RANS simulations of 2-D airfoils, providing comparisons with both single-GPU implementation and node-based graph convolution networks (GCNs). The DS-MPNN model demonstrates comparable accuracy to single-GPU implementation, can accommodate a significantly larger number of nodes compared to the single-GPU variant (S-MPNN), and significantly outperforms the node-based GCN. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15106 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sampling-based Distributed Training with Message Passing Neural Network Kakka, Priyesh Nidhan, Sheel Ranade, Rishikesh Pathak, Jay MacArt, Jonathan F. Machine Learning Distributed, Parallel, and Cluster Computing Fluid Dynamics In this study, we introduce a domain-decomposition-based distributed training and inference approach for message-passing neural networks (MPNN). Our objective is to address the challenge of scaling edge-based graph neural networks as the number of nodes increases. Through our distributed training approach, coupled with Nyström-approximation sampling techniques, we present a scalable graph neural network, referred to as DS-MPNN (D and S standing for distributed and sampled, respectively), capable of scaling up to $O(10^5)$ nodes. We validate our sampling and distributed training approach on two cases: (a) a Darcy flow dataset and (b) steady RANS simulations of 2-D airfoils, providing comparisons with both single-GPU implementation and node-based graph convolution networks (GCNs). The DS-MPNN model demonstrates comparable accuracy to single-GPU implementation, can accommodate a significantly larger number of nodes compared to the single-GPU variant (S-MPNN), and significantly outperforms the node-based GCN. |
| title | Sampling-based Distributed Training with Message Passing Neural Network |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Fluid Dynamics |
| url | https://arxiv.org/abs/2402.15106 |