A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910335545376768 |
|---|---|
| author | Li, Jintang Zhang, Huizhe Wu, Ruofan Zhu, Zulun Wang, Baokun Meng, Changhua Zheng, Zibin Chen, Liang |
| author_facet | Li, Jintang Zhang, Huizhe Wu, Ruofan Zhu, Zulun Wang, Baokun Meng, Changhua Zheng, Zibin Chen, Liang |
| contents | While contrastive self-supervised learning has become the de-facto learning paradigm for graph neural networks, the pursuit of higher task accuracy requires a larger hidden dimensionality to learn informative and discriminative full-precision representations, raising concerns about computation, memory footprint, and energy consumption burden (largely overlooked) for real-world applications. This work explores a promising direction for graph contrastive learning (GCL) with spiking neural networks (SNNs), which leverage sparse and binary characteristics to learn more biologically plausible and compact representations. We propose SpikeGCL, a novel GCL framework to learn binarized 1-bit representations for graphs, making balanced trade-offs between efficiency and performance. We provide theoretical guarantees to demonstrate that SpikeGCL has comparable expressiveness with its full-precision counterparts. Experimental results demonstrate that, with nearly 32x representation storage compression, SpikeGCL is either comparable to or outperforms many fancy state-of-the-art supervised and self-supervised methods across several graph benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_19306 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks Li, Jintang Zhang, Huizhe Wu, Ruofan Zhu, Zulun Wang, Baokun Meng, Changhua Zheng, Zibin Chen, Liang Neural and Evolutionary Computing Artificial Intelligence Machine Learning While contrastive self-supervised learning has become the de-facto learning paradigm for graph neural networks, the pursuit of higher task accuracy requires a larger hidden dimensionality to learn informative and discriminative full-precision representations, raising concerns about computation, memory footprint, and energy consumption burden (largely overlooked) for real-world applications. This work explores a promising direction for graph contrastive learning (GCL) with spiking neural networks (SNNs), which leverage sparse and binary characteristics to learn more biologically plausible and compact representations. We propose SpikeGCL, a novel GCL framework to learn binarized 1-bit representations for graphs, making balanced trade-offs between efficiency and performance. We provide theoretical guarantees to demonstrate that SpikeGCL has comparable expressiveness with its full-precision counterparts. Experimental results demonstrate that, with nearly 32x representation storage compression, SpikeGCL is either comparable to or outperforms many fancy state-of-the-art supervised and self-supervised methods across several graph benchmarks. |
| title | A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks |
| topic | Neural and Evolutionary Computing Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2305.19306 |