DYMAG: Rethinking Message Passing Using Dynamical-systems-based Waveforms
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910969212436480 |
|---|---|
| author | Bhaskar, Dhananjay Sun, Xingzhi Zhang, Yanlei Xu, Charles Afrasiyabi, Arman Viswanath, Siddharth Fasina, Oluwadamilola Nickel, Maximilian Wolf, Guy Perlmutter, Michael Krishnaswamy, Smita |
| author_facet | Bhaskar, Dhananjay Sun, Xingzhi Zhang, Yanlei Xu, Charles Afrasiyabi, Arman Viswanath, Siddharth Fasina, Oluwadamilola Nickel, Maximilian Wolf, Guy Perlmutter, Michael Krishnaswamy, Smita |
| contents | We present DYMAG, a graph neural network based on a novel form of message aggregation. Standard message-passing neural networks, which often aggregate local neighbors via mean-aggregation, can be regarded as convolving with a simple rectangular waveform which is non-zero only on 1-hop neighbors of every vertex. Here, we go beyond such local averaging. We will convolve the node features with more sophisticated waveforms generated using dynamics such as the heat equation, wave equation, and the Sprott model (an example of chaotic dynamics). Furthermore, we use snapshots of these dynamics at different time points to create waveforms at many effective scales. Theoretically, we show that these dynamic waveforms can capture salient information about the graph including connected components, connectivity, and cycle structures even with no features. Empirically, we test DYMAG on both real and synthetic benchmarks to establish that DYMAG outperforms baseline models on recovery of graph persistence, generating parameters of random graphs, as well as property prediction for proteins, molecules and materials. Our code is available at https://github.com/KrishnaswamyLab/DYMAG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_09924 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DYMAG: Rethinking Message Passing Using Dynamical-systems-based Waveforms Bhaskar, Dhananjay Sun, Xingzhi Zhang, Yanlei Xu, Charles Afrasiyabi, Arman Viswanath, Siddharth Fasina, Oluwadamilola Nickel, Maximilian Wolf, Guy Perlmutter, Michael Krishnaswamy, Smita Machine Learning Signal Processing We present DYMAG, a graph neural network based on a novel form of message aggregation. Standard message-passing neural networks, which often aggregate local neighbors via mean-aggregation, can be regarded as convolving with a simple rectangular waveform which is non-zero only on 1-hop neighbors of every vertex. Here, we go beyond such local averaging. We will convolve the node features with more sophisticated waveforms generated using dynamics such as the heat equation, wave equation, and the Sprott model (an example of chaotic dynamics). Furthermore, we use snapshots of these dynamics at different time points to create waveforms at many effective scales. Theoretically, we show that these dynamic waveforms can capture salient information about the graph including connected components, connectivity, and cycle structures even with no features. Empirically, we test DYMAG on both real and synthetic benchmarks to establish that DYMAG outperforms baseline models on recovery of graph persistence, generating parameters of random graphs, as well as property prediction for proteins, molecules and materials. Our code is available at https://github.com/KrishnaswamyLab/DYMAG. |
| title | DYMAG: Rethinking Message Passing Using Dynamical-systems-based Waveforms |
| topic | Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2309.09924 |