Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909182913937408 |
|---|---|
| author | Kang, Qiyu Zhao, Kai Ding, Qinxu Ji, Feng Li, Xuhao Liang, Wenfei Song, Yang Tay, Wee Peng |
| author_facet | Kang, Qiyu Zhao, Kai Ding, Qinxu Ji, Feng Li, Xuhao Liang, Wenfei Song, Yang Tay, Wee Peng |
| contents | We introduce the FRactional-Order graph Neural Dynamical network (FROND), a new continuous graph neural network (GNN) framework. Unlike traditional continuous GNNs that rely on integer-order differential equations, FROND employs the Caputo fractional derivative to leverage the non-local properties of fractional calculus. This approach enables the capture of long-term dependencies in feature updates, moving beyond the Markovian update mechanisms in conventional integer-order models and offering enhanced capabilities in graph representation learning. We offer an interpretation of the node feature updating process in FROND from a non-Markovian random walk perspective when the feature updating is particularly governed by a diffusion process. We demonstrate analytically that oversmoothing can be mitigated in this setting. Experimentally, we validate the FROND framework by comparing the fractional adaptations of various established integer-order continuous GNNs, demonstrating their consistently improved performance and underscoring the framework's potential as an effective extension to enhance traditional continuous GNNs. The code is available at \url{https://github.com/zknus/ICLR2024-FROND}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_17099 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND Kang, Qiyu Zhao, Kai Ding, Qinxu Ji, Feng Li, Xuhao Liang, Wenfei Song, Yang Tay, Wee Peng Machine Learning Neural and Evolutionary Computing We introduce the FRactional-Order graph Neural Dynamical network (FROND), a new continuous graph neural network (GNN) framework. Unlike traditional continuous GNNs that rely on integer-order differential equations, FROND employs the Caputo fractional derivative to leverage the non-local properties of fractional calculus. This approach enables the capture of long-term dependencies in feature updates, moving beyond the Markovian update mechanisms in conventional integer-order models and offering enhanced capabilities in graph representation learning. We offer an interpretation of the node feature updating process in FROND from a non-Markovian random walk perspective when the feature updating is particularly governed by a diffusion process. We demonstrate analytically that oversmoothing can be mitigated in this setting. Experimentally, we validate the FROND framework by comparing the fractional adaptations of various established integer-order continuous GNNs, demonstrating their consistently improved performance and underscoring the framework's potential as an effective extension to enhance traditional continuous GNNs. The code is available at \url{https://github.com/zknus/ICLR2024-FROND}. |
| title | Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND |
| topic | Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2404.17099 |