Model-free $H_{\infty}$ control of Itô stochastic system via off-policy reinforcement learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913257149693952 |
|---|---|
| author | Guo, Jing Guo Jing Jiang, Xiushan Zhang, Weihai |
| author_facet | Guo, Jing Guo Jing Jiang, Xiushan Zhang, Weihai |
| contents | The stochastic $H_{\infty}$ control is studied for a linear stochastic Itô system with an unknown system model. The linear stochastic $H_{\infty}$ control issue is known to be transformable into the problem of solving a so-called generalized algebraic Riccati equation (GARE), which is a nonlinear equation that is typically difficult to solve analytically. Worse, model-based techniques cannot be utilized to approximately solve a GARE when an accurate system model is unavailable or prohibitively expensive to construct in reality. To address these issues, an off-policy reinforcement learning (RL) approach is presented to learn the solution of a GARE from real system data rather than a system model; its convergence is demonstrated, and the robustness of RL to errors in the learning process is investigated. In the off-policy RL approach, the system data may be created with behavior policies rather than the target policies, which is highly significant and promising for use in actual systems. Finally, the proposed off-policy RL approach is validated on a stochastic linear F-16 aircraft system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04412 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Model-free $H_{\infty}$ control of Itô stochastic system via off-policy reinforcement learning Guo, Jing Guo Jing Jiang, Xiushan Zhang, Weihai Optimization and Control The stochastic $H_{\infty}$ control is studied for a linear stochastic Itô system with an unknown system model. The linear stochastic $H_{\infty}$ control issue is known to be transformable into the problem of solving a so-called generalized algebraic Riccati equation (GARE), which is a nonlinear equation that is typically difficult to solve analytically. Worse, model-based techniques cannot be utilized to approximately solve a GARE when an accurate system model is unavailable or prohibitively expensive to construct in reality. To address these issues, an off-policy reinforcement learning (RL) approach is presented to learn the solution of a GARE from real system data rather than a system model; its convergence is demonstrated, and the robustness of RL to errors in the learning process is investigated. In the off-policy RL approach, the system data may be created with behavior policies rather than the target policies, which is highly significant and promising for use in actual systems. Finally, the proposed off-policy RL approach is validated on a stochastic linear F-16 aircraft system. |
| title | Model-free $H_{\infty}$ control of Itô stochastic system via off-policy reinforcement learning |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2403.04412 |