Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.11792 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916949972221952 |
|---|---|
| author | Moustafa, Samir Kummer, Lorenz Fetzel, Simon Kriege, Nils M. Gansterer, Wilfried N. |
| author_facet | Moustafa, Samir Kummer, Lorenz Fetzel, Simon Kriege, Nils M. Gansterer, Wilfried N. |
| contents | Graph Neural Networks (GNNs) are powerful models for graph-structured data, with broad applications. However, the interplay between GNN parameter optimization, expressivity, and generalization remains poorly understood. We address this by introducing an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and by analyzing the effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization. Our learnable projection method surpasses the state-of-the-art PCA-based approach, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. We further show that architecture, sparsification, and optimizer's preconditioning significantly impact the GNN optimization landscape and their training process and final prediction performance. These insights contribute to developing more efficient designs of GNN architectures and training strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_11792 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Visualization and Analysis of the Loss Landscape in Graph Neural Networks Moustafa, Samir Kummer, Lorenz Fetzel, Simon Kriege, Nils M. Gansterer, Wilfried N. Machine Learning Graph Neural Networks (GNNs) are powerful models for graph-structured data, with broad applications. However, the interplay between GNN parameter optimization, expressivity, and generalization remains poorly understood. We address this by introducing an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and by analyzing the effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization. Our learnable projection method surpasses the state-of-the-art PCA-based approach, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. We further show that architecture, sparsification, and optimizer's preconditioning significantly impact the GNN optimization landscape and their training process and final prediction performance. These insights contribute to developing more efficient designs of GNN architectures and training strategies. |
| title | Visualization and Analysis of the Loss Landscape in Graph Neural Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.11792 |