Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912047636152320 |
|---|---|
| author | Zhang, Tianle Zhang, Yuchen Wang, Kun Wang, Kai Yang, Beining Zhang, Kaipeng Shao, Wenqi Liu, Ping Zhou, Joey Tianyi You, Yang |
| author_facet | Zhang, Tianle Zhang, Yuchen Wang, Kun Wang, Kai Yang, Beining Zhang, Kaipeng Shao, Wenqi Liu, Ping Zhou, Joey Tianyi You, Yang |
| contents | Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have raised growing concerns. As one of the most promising directions, graph condensation methods address these issues by employing gradient matching, aiming to condense the full graph into a more concise yet information-rich synthetic set. Though encouraging, these strategies primarily emphasize matching directions of the gradients, which leads to deviations in the training trajectories. Such deviations are further magnified by the differences between the condensation and evaluation phases, culminating in accumulated errors, which detrimentally affect the performance of the condensed graphs. In light of this, we propose a novel graph condensation method named \textbf{C}raf\textbf{T}ing \textbf{R}ationa\textbf{L} trajectory (\textbf{CTRL}), which offers an optimized starting point closer to the original dataset's feature distribution and a more refined strategy for gradient matching. Theoretically, CTRL can effectively neutralize the impact of accumulated errors on the performance of condensed graphs. We provide extensive experiments on various graph datasets and downstream tasks to support the effectiveness of CTRL. Code is released at https://github.com/NUS-HPC-AI-Lab/CTRL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_04924 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching Zhang, Tianle Zhang, Yuchen Wang, Kun Wang, Kai Yang, Beining Zhang, Kaipeng Shao, Wenqi Liu, Ping Zhou, Joey Tianyi You, Yang Machine Learning Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have raised growing concerns. As one of the most promising directions, graph condensation methods address these issues by employing gradient matching, aiming to condense the full graph into a more concise yet information-rich synthetic set. Though encouraging, these strategies primarily emphasize matching directions of the gradients, which leads to deviations in the training trajectories. Such deviations are further magnified by the differences between the condensation and evaluation phases, culminating in accumulated errors, which detrimentally affect the performance of the condensed graphs. In light of this, we propose a novel graph condensation method named \textbf{C}raf\textbf{T}ing \textbf{R}ationa\textbf{L} trajectory (\textbf{CTRL}), which offers an optimized starting point closer to the original dataset's feature distribution and a more refined strategy for gradient matching. Theoretically, CTRL can effectively neutralize the impact of accumulated errors on the performance of condensed graphs. We provide extensive experiments on various graph datasets and downstream tasks to support the effectiveness of CTRL. Code is released at https://github.com/NUS-HPC-AI-Lab/CTRL. |
| title | Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.04924 |