Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908276723023872 |
|---|---|
| author | Kang, Qiyu Li, Xuhao Zhao, Kai Cui, Wenjun Zhao, Yanan Deng, Weihua Tay, Wee Peng |
| author_facet | Kang, Qiyu Li, Xuhao Zhao, Kai Cui, Wenjun Zhao, Yanan Deng, Weihua Tay, Wee Peng |
| contents | Fractional-order differential equations (FDEs) enhance traditional differential equations by extending the order of differential operators from integers to real numbers, offering greater flexibility in modeling complex dynamical systems with nonlocal characteristics. Recent progress at the intersection of FDEs and deep learning has catalyzed a new wave of innovative models, demonstrating the potential to address challenges such as graph representation learning. However, training neural FDEs has primarily relied on direct differentiation through forward-pass operations in FDE numerical solvers, leading to increased memory usage and computational complexity, particularly in large-scale applications. To address these challenges, we propose a scalable adjoint backpropagation method for training neural FDEs by solving an augmented FDE backward in time, which substantially reduces memory requirements. This approach provides a practical neural FDE toolbox and holds considerable promise for diverse applications. We demonstrate the effectiveness of our method in several tasks, achieving performance comparable to baseline models while significantly reducing computational overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16666 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation Kang, Qiyu Li, Xuhao Zhao, Kai Cui, Wenjun Zhao, Yanan Deng, Weihua Tay, Wee Peng Machine Learning Fractional-order differential equations (FDEs) enhance traditional differential equations by extending the order of differential operators from integers to real numbers, offering greater flexibility in modeling complex dynamical systems with nonlocal characteristics. Recent progress at the intersection of FDEs and deep learning has catalyzed a new wave of innovative models, demonstrating the potential to address challenges such as graph representation learning. However, training neural FDEs has primarily relied on direct differentiation through forward-pass operations in FDE numerical solvers, leading to increased memory usage and computational complexity, particularly in large-scale applications. To address these challenges, we propose a scalable adjoint backpropagation method for training neural FDEs by solving an augmented FDE backward in time, which substantially reduces memory requirements. This approach provides a practical neural FDE toolbox and holds considerable promise for diverse applications. We demonstrate the effectiveness of our method in several tasks, achieving performance comparable to baseline models while significantly reducing computational overhead. |
| title | Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.16666 |