Systematic construction of continuous-time neural networks for linear dynamical systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911811836575744 |
|---|---|
| author | Datar, Chinmay Datar, Adwait Dietrich, Felix Schilders, Wil |
| author_facet | Datar, Chinmay Datar, Adwait Dietrich, Felix Schilders, Wil |
| contents | Discovering a suitable neural network architecture for modeling complex dynamical systems poses a formidable challenge, often involving extensive trial and error and navigation through a high-dimensional hyper-parameter space. In this paper, we discuss a systematic approach to constructing neural architectures for modeling a subclass of dynamical systems, namely, Linear Time-Invariant (LTI) systems. We use a variant of continuous-time neural networks in which the output of each neuron evolves continuously as a solution of a first-order or second-order Ordinary Differential Equation (ODE). Instead of deriving the network architecture and parameters from data, we propose a gradient-free algorithm to compute sparse architecture and network parameters directly from the given LTI system, leveraging its properties. We bring forth a novel neural architecture paradigm featuring horizontal hidden layers and provide insights into why employing conventional neural architectures with vertical hidden layers may not be favorable. We also provide an upper bound on the numerical errors of our neural networks. Finally, we demonstrate the high accuracy of our constructed networks on three numerical examples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16215 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Systematic construction of continuous-time neural networks for linear dynamical systems Datar, Chinmay Datar, Adwait Dietrich, Felix Schilders, Wil Machine Learning Numerical Analysis Dynamical Systems 93B17, 65L70, 68T07 I.2.m; G.1.3; G.1.7 Discovering a suitable neural network architecture for modeling complex dynamical systems poses a formidable challenge, often involving extensive trial and error and navigation through a high-dimensional hyper-parameter space. In this paper, we discuss a systematic approach to constructing neural architectures for modeling a subclass of dynamical systems, namely, Linear Time-Invariant (LTI) systems. We use a variant of continuous-time neural networks in which the output of each neuron evolves continuously as a solution of a first-order or second-order Ordinary Differential Equation (ODE). Instead of deriving the network architecture and parameters from data, we propose a gradient-free algorithm to compute sparse architecture and network parameters directly from the given LTI system, leveraging its properties. We bring forth a novel neural architecture paradigm featuring horizontal hidden layers and provide insights into why employing conventional neural architectures with vertical hidden layers may not be favorable. We also provide an upper bound on the numerical errors of our neural networks. Finally, we demonstrate the high accuracy of our constructed networks on three numerical examples. |
| title | Systematic construction of continuous-time neural networks for linear dynamical systems |
| topic | Machine Learning Numerical Analysis Dynamical Systems 93B17, 65L70, 68T07 I.2.m; G.1.3; G.1.7 |
| url | https://arxiv.org/abs/2403.16215 |