Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918164722352128 |
|---|---|
| author | Deng, Gangda Yang, Yuxin Akgül, Ömer Faruk Zeng, Hanqing Xia, Yinglong Kannan, Rajgopal Prasanna, Viktor |
| author_facet | Deng, Gangda Yang, Yuxin Akgül, Ömer Faruk Zeng, Hanqing Xia, Yinglong Kannan, Rajgopal Prasanna, Viktor |
| contents | Graph Neural Networks (GNNs) have become essential tools for learning on relational data, yet the performance of a single GNN is often limited by the heterogeneity present in real-world graphs. Recent advances in Mixture-of-Experts (MoE) frameworks demonstrate that assembling multiple, explicitly diverse GNNs with distinct generalization patterns can significantly improve performance. In this work, we present the first systematic empirical study of expert-level diversification techniques for GNN ensembles. Evaluating 20 diversification strategies -- including random re-initialization, hyperparameter tuning, architectural variation, directionality modeling, and training data partitioning -- across 14 node classification benchmarks, we construct and analyze over 200 ensemble variants. Our comprehensive evaluation examines each technique in terms of expert diversity, complementarity, and ensemble performance. We also uncovers mechanistic insights into training maximally diverse experts. These findings provide actionable guidance for expert training and the design of effective MoE frameworks on graph data. Our code is available at https://github.com/Hydrapse/bench-gnn-diversification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_18370 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study Deng, Gangda Yang, Yuxin Akgül, Ömer Faruk Zeng, Hanqing Xia, Yinglong Kannan, Rajgopal Prasanna, Viktor Machine Learning Graph Neural Networks (GNNs) have become essential tools for learning on relational data, yet the performance of a single GNN is often limited by the heterogeneity present in real-world graphs. Recent advances in Mixture-of-Experts (MoE) frameworks demonstrate that assembling multiple, explicitly diverse GNNs with distinct generalization patterns can significantly improve performance. In this work, we present the first systematic empirical study of expert-level diversification techniques for GNN ensembles. Evaluating 20 diversification strategies -- including random re-initialization, hyperparameter tuning, architectural variation, directionality modeling, and training data partitioning -- across 14 node classification benchmarks, we construct and analyze over 200 ensemble variants. Our comprehensive evaluation examines each technique in terms of expert diversity, complementarity, and ensemble performance. We also uncovers mechanistic insights into training maximally diverse experts. These findings provide actionable guidance for expert training and the design of effective MoE frameworks on graph data. Our code is available at https://github.com/Hydrapse/bench-gnn-diversification. |
| title | Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.18370 |