Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929337511444480 |
|---|---|
| author | Liu, Yujing Wu, Zongqian Lu, Zhengyu Nie, Ci Wen, Guoqiu Hu, Ping Zhu, Xiaofeng |
| author_facet | Liu, Yujing Wu, Zongqian Lu, Zhengyu Nie, Ci Wen, Guoqiu Hu, Ping Zhu, Xiaofeng |
| contents | Previous graph neural networks (GNNs) usually assume that the graph data is with clean labels for representation learning, but it is not true in real applications. In this paper, we propose a new multi-teacher distillation method based on bi-level optimization (namely BO-NNC), to conduct noisy node classification on the graph data. Specifically, we first employ multiple self-supervised learning methods to train diverse teacher models, and then aggregate their predictions through a teacher weight matrix. Furthermore, we design a new bi-level optimization strategy to dynamically adjust the teacher weight matrix based on the training progress of the student model. Finally, we design a label improvement module to improve the label quality. Extensive experimental results on real datasets show that our method achieves the best results compared to state-of-the-art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_17875 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation Liu, Yujing Wu, Zongqian Lu, Zhengyu Nie, Ci Wen, Guoqiu Hu, Ping Zhu, Xiaofeng Machine Learning Previous graph neural networks (GNNs) usually assume that the graph data is with clean labels for representation learning, but it is not true in real applications. In this paper, we propose a new multi-teacher distillation method based on bi-level optimization (namely BO-NNC), to conduct noisy node classification on the graph data. Specifically, we first employ multiple self-supervised learning methods to train diverse teacher models, and then aggregate their predictions through a teacher weight matrix. Furthermore, we design a new bi-level optimization strategy to dynamically adjust the teacher weight matrix based on the training progress of the student model. Finally, we design a label improvement module to improve the label quality. Extensive experimental results on real datasets show that our method achieves the best results compared to state-of-the-art methods. |
| title | Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2404.17875 |