Primal-dual policy learning for mean-field stochastic LQR problem
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917133926006784 |
|---|---|
| author | Jiang, Xiushan Wang, Dong Zhang, Weihai Ho, Daniel W. C. Wu, Yuanqing |
| author_facet | Jiang, Xiushan Wang, Dong Zhang, Weihai Ho, Daniel W. C. Wu, Yuanqing |
| contents | Integrating data-driven techniques with mechanism-driven insights has recently gained popularity as a powerful learning approach to solving traditional LQR problems for designing intelligent controllers in complex dynamic systems. However, the theoretical understanding of various reinforcement learning algorithms needs further exploration to enhance their efficiency and safety. In this article, by means of primal-dual optimization tools, we study the partially model-free design of the mean-field stochastic LQR (MF-SLQR) controller using a policy learning approach. Firstly, by designing appropriate optimizing variables, the considered MF-SLQR problem is transformed into a new static nonconvex constrained optimization problem with equivalence preserved in certain senses. After that, the equivalent formulation of the duality results is constructed via finding the solution of the generalized Lyapunov equation. Then, the strong duality is analyzed, based on which we establish a primal-dual algorithm by Karush-Kuhn-Tucker conditions. More importantly, a partially model-free implementation is also presented, which has a direct connection with the classical policy iteration algorithm. Finally, we use a high-dimensional example to validate our methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08205 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Primal-dual policy learning for mean-field stochastic LQR problem Jiang, Xiushan Wang, Dong Zhang, Weihai Ho, Daniel W. C. Wu, Yuanqing Optimization and Control Integrating data-driven techniques with mechanism-driven insights has recently gained popularity as a powerful learning approach to solving traditional LQR problems for designing intelligent controllers in complex dynamic systems. However, the theoretical understanding of various reinforcement learning algorithms needs further exploration to enhance their efficiency and safety. In this article, by means of primal-dual optimization tools, we study the partially model-free design of the mean-field stochastic LQR (MF-SLQR) controller using a policy learning approach. Firstly, by designing appropriate optimizing variables, the considered MF-SLQR problem is transformed into a new static nonconvex constrained optimization problem with equivalence preserved in certain senses. After that, the equivalent formulation of the duality results is constructed via finding the solution of the generalized Lyapunov equation. Then, the strong duality is analyzed, based on which we establish a primal-dual algorithm by Karush-Kuhn-Tucker conditions. More importantly, a partially model-free implementation is also presented, which has a direct connection with the classical policy iteration algorithm. Finally, we use a high-dimensional example to validate our methods. |
| title | Primal-dual policy learning for mean-field stochastic LQR problem |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2512.08205 |