Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917730067677184 |
|---|---|
| author | Zhao, Huanjing Yang, Beining Cen, Yukuo Ren, Junyu Zhang, Chenhui Dong, Yuxiao Kharlamov, Evgeny Zhao, Shu Tang, Jie |
| author_facet | Zhao, Huanjing Yang, Beining Cen, Yukuo Ren, Junyu Zhang, Chenhui Dong, Yuxiao Kharlamov, Evgeny Zhao, Shu Tang, Jie |
| contents | The text-attributed graph (TAG) is one kind of important real-world graph-structured data with each node associated with raw texts. For TAGs, traditional few-shot node classification methods directly conduct training on the pre-processed node features and do not consider the raw texts. The performance is highly dependent on the choice of the feature pre-processing method. In this paper, we propose P2TAG, a framework designed for few-shot node classification on TAGs with graph pre-training and prompting. P2TAG first pre-trains the language model (LM) and graph neural network (GNN) on TAGs with self-supervised loss. To fully utilize the ability of language models, we adapt the masked language modeling objective for our framework. The pre-trained model is then used for the few-shot node classification with a mixed prompt method, which simultaneously considers both text and graph information. We conduct experiments on six real-world TAGs, including paper citation networks and product co-purchasing networks. Experimental results demonstrate that our proposed framework outperforms existing graph few-shot learning methods on these datasets with +18.98% ~ +35.98% improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_15431 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs Zhao, Huanjing Yang, Beining Cen, Yukuo Ren, Junyu Zhang, Chenhui Dong, Yuxiao Kharlamov, Evgeny Zhao, Shu Tang, Jie Social and Information Networks Artificial Intelligence Machine Learning The text-attributed graph (TAG) is one kind of important real-world graph-structured data with each node associated with raw texts. For TAGs, traditional few-shot node classification methods directly conduct training on the pre-processed node features and do not consider the raw texts. The performance is highly dependent on the choice of the feature pre-processing method. In this paper, we propose P2TAG, a framework designed for few-shot node classification on TAGs with graph pre-training and prompting. P2TAG first pre-trains the language model (LM) and graph neural network (GNN) on TAGs with self-supervised loss. To fully utilize the ability of language models, we adapt the masked language modeling objective for our framework. The pre-trained model is then used for the few-shot node classification with a mixed prompt method, which simultaneously considers both text and graph information. We conduct experiments on six real-world TAGs, including paper citation networks and product co-purchasing networks. Experimental results demonstrate that our proposed framework outperforms existing graph few-shot learning methods on these datasets with +18.98% ~ +35.98% improvements. |
| title | Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs |
| topic | Social and Information Networks Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.15431 |