StructComp: Substituting Propagation with Structural Compression in Training Graph Contrastive Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913345152483328 |
|---|---|
| author | Zhang, Shengzhong Yang, Wenjie Cao, Xinyuan Zhang, Hongwei Huang, Zengfeng |
| author_facet | Zhang, Shengzhong Yang, Wenjie Cao, Xinyuan Zhang, Hongwei Huang, Zengfeng |
| contents | Graph contrastive learning (GCL) has become a powerful tool for learning graph data, but its scalability remains a significant challenge. In this work, we propose a simple yet effective training framework called Structural Compression (StructComp) to address this issue. Inspired by a sparse low-rank approximation on the diffusion matrix, StructComp trains the encoder with the compressed nodes. This allows the encoder not to perform any message passing during the training stage, and significantly reduces the number of sample pairs in the contrastive loss. We theoretically prove that the original GCL loss can be approximated with the contrastive loss computed by StructComp. Moreover, StructComp can be regarded as an additional regularization term for GCL models, resulting in a more robust encoder. Empirical studies on various datasets show that StructComp greatly reduces the time and memory consumption while improving model performance compared to the vanilla GCL models and scalable training methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_04865 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | StructComp: Substituting Propagation with Structural Compression in Training Graph Contrastive Learning Zhang, Shengzhong Yang, Wenjie Cao, Xinyuan Zhang, Hongwei Huang, Zengfeng Machine Learning Graph contrastive learning (GCL) has become a powerful tool for learning graph data, but its scalability remains a significant challenge. In this work, we propose a simple yet effective training framework called Structural Compression (StructComp) to address this issue. Inspired by a sparse low-rank approximation on the diffusion matrix, StructComp trains the encoder with the compressed nodes. This allows the encoder not to perform any message passing during the training stage, and significantly reduces the number of sample pairs in the contrastive loss. We theoretically prove that the original GCL loss can be approximated with the contrastive loss computed by StructComp. Moreover, StructComp can be regarded as an additional regularization term for GCL models, resulting in a more robust encoder. Empirical studies on various datasets show that StructComp greatly reduces the time and memory consumption while improving model performance compared to the vanilla GCL models and scalable training methods. |
| title | StructComp: Substituting Propagation with Structural Compression in Training Graph Contrastive Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2312.04865 |