Out-of-Distribution Graph Models Merging
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912969881812992 |
|---|---|
| author | Wang, Yidi Qiao, Ziyue Gu, Jiawei Zheng, Xubin Wang, Pengyang Pei, Xiaobing Luo, Xiao |
| author_facet | Wang, Yidi Qiao, Ziyue Gu, Jiawei Zheng, Xubin Wang, Pengyang Pei, Xiaobing Luo, Xiao |
| contents | This paper studies a novel problem of out-of-distribution graph models merging, which aims to construct a generalized model from multiple graph models pre-trained on different domains with distribution discrepancy. This problem is challenging because of the difficulty in learning domain-invariant knowledge implicitly in model parameters and consolidating expertise from potentially heterogeneous GNN backbones. In this work, we propose a graph generation strategy that instantiates the mixture distribution of multiple domains. Then, we merge and fine-tune the pre-trained graph models via a MoE module and a masking mechanism for generalized adaptation. Our framework is architecture-agnostic and can operate without any source/target domain data. Both theoretical analysis and experimental results demonstrate the effectiveness of our approach in addressing the model generalization problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_03674 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Out-of-Distribution Graph Models Merging Wang, Yidi Qiao, Ziyue Gu, Jiawei Zheng, Xubin Wang, Pengyang Pei, Xiaobing Luo, Xiao Machine Learning This paper studies a novel problem of out-of-distribution graph models merging, which aims to construct a generalized model from multiple graph models pre-trained on different domains with distribution discrepancy. This problem is challenging because of the difficulty in learning domain-invariant knowledge implicitly in model parameters and consolidating expertise from potentially heterogeneous GNN backbones. In this work, we propose a graph generation strategy that instantiates the mixture distribution of multiple domains. Then, we merge and fine-tune the pre-trained graph models via a MoE module and a masking mechanism for generalized adaptation. Our framework is architecture-agnostic and can operate without any source/target domain data. Both theoretical analysis and experimental results demonstrate the effectiveness of our approach in addressing the model generalization problem. |
| title | Out-of-Distribution Graph Models Merging |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.03674 |