Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912787643498496 |
|---|---|
| author | Guo, Jing Jiang, Xiushan Zhang, Weihai |
| author_facet | Guo, Jing Jiang, Xiushan Zhang, Weihai |
| contents | This paper investigates a model-free solution to the stochastic linear quadratic regulation (LQR) problem for linear discrete-time systems with both multiplicative and additive noises. We formulate the stochastic LQR problem as a nonconvex optimization problem and rigorously analyze its dual problem structure. By exploiting the inherent convexity of the dual problem and analyzing Karush-Kuhn-Tucker conditions with respect to optimality in convex optimization, we establish an explicit relationship between the optimal point of the dual problem and the parameters of the associated Q-function. This theoretical insight, combined with the technique of the matrix direct sum, makes it possible to develop a novel model-free sample-efficient, non-iterative semidefinite programming algorithm that directly estimates optimal control gain without requiring an initial stabilizing controller, or noises measurability. The robustness of the model-free SDP method to errors is investigated. Our approach provides a new optimization-theoretic framework for understanding Q-learning algorithms while advancing the theoretical foundations of reinforcement learning in stochastic optimal control. Numerical validation on a pulse-width modulated inverter system demonstrates the algorithm's effectiveness, particularly in achieving a single-step non-iterative solution without hyper-parameter tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_20911 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming Guo, Jing Jiang, Xiushan Zhang, Weihai Optimization and Control This paper investigates a model-free solution to the stochastic linear quadratic regulation (LQR) problem for linear discrete-time systems with both multiplicative and additive noises. We formulate the stochastic LQR problem as a nonconvex optimization problem and rigorously analyze its dual problem structure. By exploiting the inherent convexity of the dual problem and analyzing Karush-Kuhn-Tucker conditions with respect to optimality in convex optimization, we establish an explicit relationship between the optimal point of the dual problem and the parameters of the associated Q-function. This theoretical insight, combined with the technique of the matrix direct sum, makes it possible to develop a novel model-free sample-efficient, non-iterative semidefinite programming algorithm that directly estimates optimal control gain without requiring an initial stabilizing controller, or noises measurability. The robustness of the model-free SDP method to errors is investigated. Our approach provides a new optimization-theoretic framework for understanding Q-learning algorithms while advancing the theoretical foundations of reinforcement learning in stochastic optimal control. Numerical validation on a pulse-width modulated inverter system demonstrates the algorithm's effectiveness, particularly in achieving a single-step non-iterative solution without hyper-parameter tuning. |
| title | Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2512.20911 |