Adaptive Control of Positive Systems with Application to Learning SSP
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908327700594688 |
|---|---|
| author | Bencherki, Fethi Rantzer, Anders |
| author_facet | Bencherki, Fethi Rantzer, Anders |
| contents | An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a "data-driven algebraic equation" constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17012 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptive Control of Positive Systems with Application to Learning SSP Bencherki, Fethi Rantzer, Anders Optimization and Control Systems and Control An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a "data-driven algebraic equation" constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment. |
| title | Adaptive Control of Positive Systems with Application to Learning SSP |
| topic | Optimization and Control Systems and Control |
| url | https://arxiv.org/abs/2412.17012 |