Scaling policy iteration based reinforcement learning for unknown discrete-time linear systems
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910695737524224 |
|---|---|
| author | Pang, Zhen Tang, Shengda Cheng, Jun He, Shuping |
| author_facet | Pang, Zhen Tang, Shengda Cheng, Jun He, Shuping |
| contents | In optimal control problem, policy iteration (PI) is a powerful reinforcement learning (RL) tool used for designing optimal controller for the linear systems. However, the need for an initial stabilizing control policy significantly limits its applicability. To address this constraint, this paper proposes a novel scaling technique, which progressively brings a sequence of stable scaled systems closer to the original system, enabling the acquisition of stable control gain. Based on the designed scaling update law, we develop model-based and model-free scaling policy iteration (SPI) algorithms for solving the optimal control problem for discrete-time linear systems, in both known and completely unknown system dynamics scenarios. Unlike existing works on PI based RL, the SPI algorithms do not necessitate an initial stabilizing gain to initialize the algorithms, they can achieve the optimal control under any initial control gain. Finally, the numerical results validate the theoretical findings and confirm the effectiveness of the algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_07825 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scaling policy iteration based reinforcement learning for unknown discrete-time linear systems Pang, Zhen Tang, Shengda Cheng, Jun He, Shuping Optimization and Control In optimal control problem, policy iteration (PI) is a powerful reinforcement learning (RL) tool used for designing optimal controller for the linear systems. However, the need for an initial stabilizing control policy significantly limits its applicability. To address this constraint, this paper proposes a novel scaling technique, which progressively brings a sequence of stable scaled systems closer to the original system, enabling the acquisition of stable control gain. Based on the designed scaling update law, we develop model-based and model-free scaling policy iteration (SPI) algorithms for solving the optimal control problem for discrete-time linear systems, in both known and completely unknown system dynamics scenarios. Unlike existing works on PI based RL, the SPI algorithms do not necessitate an initial stabilizing gain to initialize the algorithms, they can achieve the optimal control under any initial control gain. Finally, the numerical results validate the theoretical findings and confirm the effectiveness of the algorithms. |
| title | Scaling policy iteration based reinforcement learning for unknown discrete-time linear systems |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2411.07825 |