Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.12526 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912381396844544 |
|---|---|
| author | Panyshev, Alexander Vinichenko, Dmitry Travkin, Oleg Alferov, Roman Zaytsev, Alexey |
| author_facet | Panyshev, Alexander Vinichenko, Dmitry Travkin, Oleg Alferov, Roman Zaytsev, Alexey |
| contents | Temporal Graph Networks (TGNs), while being accurate, face significant training inefficiencies due to irregular supervision signals in dynamic graphs, which induce sparse gradient updates. We first theoretically establish that aggregating historical node interactions into pseudo-labels reduces gradient variance, accelerating convergence. Building on this analysis, we propose History-Averaged Labels (HAL), a method that dynamically enriches training batches with pseudo-targets derived from historical label distributions. HAL ensures continuous parameter updates without architectural modifications by converting idle computation into productive learning steps. Experiments on the Temporal Graph Benchmark (TGB) validate our findings and an assumption about slow change of user preferences: HAL accelerates TGNv2 training by up to 15x while maintaining competitive performance. Thus, this work offers an efficient, lightweight, architecture-agnostic, and theoretically motivated solution to label sparsity in temporal graph learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_12526 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Never Skip a Batch: Continuous Training of Temporal GNNs via Adaptive Pseudo-Supervision Panyshev, Alexander Vinichenko, Dmitry Travkin, Oleg Alferov, Roman Zaytsev, Alexey Machine Learning Temporal Graph Networks (TGNs), while being accurate, face significant training inefficiencies due to irregular supervision signals in dynamic graphs, which induce sparse gradient updates. We first theoretically establish that aggregating historical node interactions into pseudo-labels reduces gradient variance, accelerating convergence. Building on this analysis, we propose History-Averaged Labels (HAL), a method that dynamically enriches training batches with pseudo-targets derived from historical label distributions. HAL ensures continuous parameter updates without architectural modifications by converting idle computation into productive learning steps. Experiments on the Temporal Graph Benchmark (TGB) validate our findings and an assumption about slow change of user preferences: HAL accelerates TGNv2 training by up to 15x while maintaining competitive performance. Thus, this work offers an efficient, lightweight, architecture-agnostic, and theoretically motivated solution to label sparsity in temporal graph learning. |
| title | Never Skip a Batch: Continuous Training of Temporal GNNs via Adaptive Pseudo-Supervision |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.12526 |