Reinforcement Learning-based Control of Nonlinear Systems using Carleman Approximation: Structured and Unstructured Designs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911980994953216 |
|---|---|
| author | Kar, Jishnudeep Bai, He Chakrabortty, Aranya |
| author_facet | Kar, Jishnudeep Bai, He Chakrabortty, Aranya |
| contents | We develop data-driven reinforcement learning (RL) control designs for input-affine nonlinear systems. We use Carleman linearization to express the state-space representation of the nonlinear dynamical model in the Carleman space, and develop a real-time algorithm that can learn nonlinear state-feedback controllers using state and input measurements in the infinite-dimensional Carleman space. Thereafter, we study the practicality of having a finite-order truncation of the control signal, followed by its closed-loop stability analysis. Finally, we develop two additional designs that can learn structured as well as sparse representations of the RL-based nonlinear controller, and provide theoretical conditions for ensuring their closed-loop stability. We present numerical examples to show how our proposed method generates closed-loop responses that are close to the optimal performance of the nonlinear plant. We also compare our designs to other data-driven nonlinear RL control methods such as those based on neural networks, and illustrate their relative advantages and drawbacks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_10864 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Reinforcement Learning-based Control of Nonlinear Systems using Carleman Approximation: Structured and Unstructured Designs Kar, Jishnudeep Bai, He Chakrabortty, Aranya Systems and Control We develop data-driven reinforcement learning (RL) control designs for input-affine nonlinear systems. We use Carleman linearization to express the state-space representation of the nonlinear dynamical model in the Carleman space, and develop a real-time algorithm that can learn nonlinear state-feedback controllers using state and input measurements in the infinite-dimensional Carleman space. Thereafter, we study the practicality of having a finite-order truncation of the control signal, followed by its closed-loop stability analysis. Finally, we develop two additional designs that can learn structured as well as sparse representations of the RL-based nonlinear controller, and provide theoretical conditions for ensuring their closed-loop stability. We present numerical examples to show how our proposed method generates closed-loop responses that are close to the optimal performance of the nonlinear plant. We also compare our designs to other data-driven nonlinear RL control methods such as those based on neural networks, and illustrate their relative advantages and drawbacks. |
| title | Reinforcement Learning-based Control of Nonlinear Systems using Carleman Approximation: Structured and Unstructured Designs |
| topic | Systems and Control |
| url | https://arxiv.org/abs/2302.10864 |