EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916354254176256 |
|---|---|
| author | Wang, Haohui Mao, Yuzhen Yan, Yujun Yang, Yaoqing Sun, Jianhui Choi, Kevin Veeramani, Balaji Hu, Alison Bowen, Edward Cody, Tyler Zhou, Dawei |
| author_facet | Wang, Haohui Mao, Yuzhen Yan, Yujun Yang, Yaoqing Sun, Jianhui Choi, Kevin Veeramani, Balaji Hu, Alison Bowen, Edward Cody, Tyler Zhou, Dawei |
| contents | Non-IID transfer learning on graphs is crucial in many high-stakes domains. The majority of existing works assume stationary distribution for both source and target domains. However, real-world graphs are intrinsically dynamic, presenting challenges in terms of domain evolution and dynamic discrepancy between source and target domains. To bridge the gap, we shift the problem to the dynamic setting and pose the question: given the label-rich source graphs and the label-scarce target graphs both observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer it, we propose a generalization bound for dynamic non-IID transfer learning on graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target graphs. Inspired by the theoretical results, we introduce a novel generic framework named EvoluNet. It leverages a transformer-based temporal encoding module to model temporal information of the evolving domains and then uses a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, EvoluNet outperforms the state-of-the-art models by up to 12.1%, demonstrating its effectiveness in transferring knowledge from dynamic source graphs to dynamic target graphs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_00664 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs Wang, Haohui Mao, Yuzhen Yan, Yujun Yang, Yaoqing Sun, Jianhui Choi, Kevin Veeramani, Balaji Hu, Alison Bowen, Edward Cody, Tyler Zhou, Dawei Machine Learning Non-IID transfer learning on graphs is crucial in many high-stakes domains. The majority of existing works assume stationary distribution for both source and target domains. However, real-world graphs are intrinsically dynamic, presenting challenges in terms of domain evolution and dynamic discrepancy between source and target domains. To bridge the gap, we shift the problem to the dynamic setting and pose the question: given the label-rich source graphs and the label-scarce target graphs both observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer it, we propose a generalization bound for dynamic non-IID transfer learning on graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target graphs. Inspired by the theoretical results, we introduce a novel generic framework named EvoluNet. It leverages a transformer-based temporal encoding module to model temporal information of the evolving domains and then uses a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, EvoluNet outperforms the state-of-the-art models by up to 12.1%, demonstrating its effectiveness in transferring knowledge from dynamic source graphs to dynamic target graphs. |
| title | EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.00664 |