iSpLib: A Library for Accelerating Graph Neural Networks using Auto-tuned Sparse Operations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910378487709696 |
|---|---|
| author | Anik, Md Saidul Hoque Badhe, Pranav Gampa, Rohit Azad, Ariful |
| author_facet | Anik, Md Saidul Hoque Badhe, Pranav Gampa, Rohit Azad, Ariful |
| contents | Core computations in Graph Neural Network (GNN) training and inference are often mapped to sparse matrix operations such as sparse-dense matrix multiplication (SpMM). These sparse operations are harder to optimize by manual tuning because their performance depends significantly on the sparsity of input graphs, GNN models, and computing platforms. To address this challenge, we present iSpLib, a PyTorch-based C++ library equipped with auto-tuned sparse operations. iSpLib expedites GNN training with a cache-enabled backpropagation that stores intermediate matrices in local caches. The library offers a user-friendly Python plug-in that allows users to take advantage of our optimized PyTorch operations out-of-the-box for any existing linear algebra-based PyTorch implementation of popular GNNs (Graph Convolution Network, GraphSAGE, Graph Inference Network, etc.) with only two lines of additional code. We demonstrate that iSpLib obtains up to 27x overall training speedup compared to the equivalent PyTorch 2.1.0 and PyTorch Geometric 2.4.0 implementations on the CPU. Our library is publicly available at https://github.com/HipGraph/iSpLib (https://doi.org/10.5281/zenodo.10806511). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14853 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | iSpLib: A Library for Accelerating Graph Neural Networks using Auto-tuned Sparse Operations Anik, Md Saidul Hoque Badhe, Pranav Gampa, Rohit Azad, Ariful Machine Learning Distributed, Parallel, and Cluster Computing Performance Core computations in Graph Neural Network (GNN) training and inference are often mapped to sparse matrix operations such as sparse-dense matrix multiplication (SpMM). These sparse operations are harder to optimize by manual tuning because their performance depends significantly on the sparsity of input graphs, GNN models, and computing platforms. To address this challenge, we present iSpLib, a PyTorch-based C++ library equipped with auto-tuned sparse operations. iSpLib expedites GNN training with a cache-enabled backpropagation that stores intermediate matrices in local caches. The library offers a user-friendly Python plug-in that allows users to take advantage of our optimized PyTorch operations out-of-the-box for any existing linear algebra-based PyTorch implementation of popular GNNs (Graph Convolution Network, GraphSAGE, Graph Inference Network, etc.) with only two lines of additional code. We demonstrate that iSpLib obtains up to 27x overall training speedup compared to the equivalent PyTorch 2.1.0 and PyTorch Geometric 2.4.0 implementations on the CPU. Our library is publicly available at https://github.com/HipGraph/iSpLib (https://doi.org/10.5281/zenodo.10806511). |
| title | iSpLib: A Library for Accelerating Graph Neural Networks using Auto-tuned Sparse Operations |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2403.14853 |