Metric-Entropy Limits on the Approximation of Nonlinear Dynamical Systems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915335588806656 |
|---|---|
| author | Pan, Yang Hutter, Clemens Bölcskei, Helmut |
| author_facet | Pan, Yang Hutter, Clemens Bölcskei, Helmut |
| contents | This paper is concerned with fundamental limits on the approximation of nonlinear dynamical systems. Specifically, we show that recurrent neural networks (RNNs) can approximate nonlinear systems -- that satisfy a Lipschitz property and forget past inputs fast enough -- in metric-entropy-optimal manner. As the sets of sequence-to-sequence mappings realized by the dynamical systems we consider are significantly more massive than function classes generally analyzed in approximation theory, a refined metric-entropy characterization is needed, namely in terms of order, type, and generalized dimension. We compute these quantities for the classes of exponentially- and polynomially Lipschitz fading-memory systems and show that RNNs can achieve them. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_01250 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Metric-Entropy Limits on the Approximation of Nonlinear Dynamical Systems Pan, Yang Hutter, Clemens Bölcskei, Helmut Machine Learning Information Theory Dynamical Systems This paper is concerned with fundamental limits on the approximation of nonlinear dynamical systems. Specifically, we show that recurrent neural networks (RNNs) can approximate nonlinear systems -- that satisfy a Lipschitz property and forget past inputs fast enough -- in metric-entropy-optimal manner. As the sets of sequence-to-sequence mappings realized by the dynamical systems we consider are significantly more massive than function classes generally analyzed in approximation theory, a refined metric-entropy characterization is needed, namely in terms of order, type, and generalized dimension. We compute these quantities for the classes of exponentially- and polynomially Lipschitz fading-memory systems and show that RNNs can achieve them. |
| title | Metric-Entropy Limits on the Approximation of Nonlinear Dynamical Systems |
| topic | Machine Learning Information Theory Dynamical Systems |
| url | https://arxiv.org/abs/2407.01250 |