Bootstrap Policy Iteration for Stochastic LQ Tracking with Multiplicative Noise
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909757301850112 |
|---|---|
| author | Chen, Jiayu Xu, Zhenhui Wang, Xinghu |
| author_facet | Chen, Jiayu Xu, Zhenhui Wang, Xinghu |
| contents | This paper studies the optimal tracking control problem for continuous-time stochastic linear systems with multiplicative noise. The solution framework involves solving a stochastic algebraic Riccati equation for the feedback gain and a Sylvester equation for the feedforward gain. To enable model-free optimal tracking, we first develop a two-phase bootstrap policy iteration (B-PI) algorithm, which bootstraps a stabilizing control gain from the trivially initialized zero-value start and proceeds with standard policy iteration. Building on this algorithm, we propose a data-driven, off-policy reinforcement learning approach that ensures convergence to the optimal feedback gain under the interval excitation condition. We further introduce a data-driven method to compute the feedforward using the obtained feedback gain. Additionally, for systems with state-dependent noise, we propose a shadow system-based optimal tracking method to eliminate the need for probing noise. The effectiveness of the proposed methods is demonstrated through numerical examples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20394 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bootstrap Policy Iteration for Stochastic LQ Tracking with Multiplicative Noise Chen, Jiayu Xu, Zhenhui Wang, Xinghu Systems and Control This paper studies the optimal tracking control problem for continuous-time stochastic linear systems with multiplicative noise. The solution framework involves solving a stochastic algebraic Riccati equation for the feedback gain and a Sylvester equation for the feedforward gain. To enable model-free optimal tracking, we first develop a two-phase bootstrap policy iteration (B-PI) algorithm, which bootstraps a stabilizing control gain from the trivially initialized zero-value start and proceeds with standard policy iteration. Building on this algorithm, we propose a data-driven, off-policy reinforcement learning approach that ensures convergence to the optimal feedback gain under the interval excitation condition. We further introduce a data-driven method to compute the feedforward using the obtained feedback gain. Additionally, for systems with state-dependent noise, we propose a shadow system-based optimal tracking method to eliminate the need for probing noise. The effectiveness of the proposed methods is demonstrated through numerical examples. |
| title | Bootstrap Policy Iteration for Stochastic LQ Tracking with Multiplicative Noise |
| topic | Systems and Control |
| url | https://arxiv.org/abs/2508.20394 |