Visualization and Analysis of the Loss Landscape in Graph Neural Networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916949972221952 |
|---|---|
| author | Moustafa, Samir Kummer, Lorenz Fetzel, Simon Kriege, Nils M. Gansterer, Wilfried N. |
| author_facet | Moustafa, Samir Kummer, Lorenz Fetzel, Simon Kriege, Nils M. Gansterer, Wilfried N. |
| contents | Graph Neural Networks (GNNs) are powerful models for graph-structured data, with broad applications. However, the interplay between GNN parameter optimization, expressivity, and generalization remains poorly understood. We address this by introducing an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and by analyzing the effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization. Our learnable projection method surpasses the state-of-the-art PCA-based approach, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. We further show that architecture, sparsification, and optimizer's preconditioning significantly impact the GNN optimization landscape and their training process and final prediction performance. These insights contribute to developing more efficient designs of GNN architectures and training strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_11792 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Visualization and Analysis of the Loss Landscape in Graph Neural Networks Moustafa, Samir Kummer, Lorenz Fetzel, Simon Kriege, Nils M. Gansterer, Wilfried N. Machine Learning Graph Neural Networks (GNNs) are powerful models for graph-structured data, with broad applications. However, the interplay between GNN parameter optimization, expressivity, and generalization remains poorly understood. We address this by introducing an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and by analyzing the effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization. Our learnable projection method surpasses the state-of-the-art PCA-based approach, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. We further show that architecture, sparsification, and optimizer's preconditioning significantly impact the GNN optimization landscape and their training process and final prediction performance. These insights contribute to developing more efficient designs of GNN architectures and training strategies. |
| title | Visualization and Analysis of the Loss Landscape in Graph Neural Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.11792 |