Stability-Informed Initialization of Neural Ordinary Differential Equations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910556609314816 |
|---|---|
| author | Westny, Theodor Mohammadi, Arman Jung, Daniel Frisk, Erik |
| author_facet | Westny, Theodor Mohammadi, Arman Jung, Daniel Frisk, Erik |
| contents | This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver's corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_15890 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Stability-Informed Initialization of Neural Ordinary Differential Equations Westny, Theodor Mohammadi, Arman Jung, Daniel Frisk, Erik Machine Learning Computer Vision and Pattern Recognition This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver's corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications. |
| title | Stability-Informed Initialization of Neural Ordinary Differential Equations |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2311.15890 |