A Globally Convergent Policy Gradient Method for Linear Quadratic Gaussian (LQG) Control
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909119625035776 |
|---|---|
| author | Sadamoto, Tomonori Nakamata, Fumiya |
| author_facet | Sadamoto, Tomonori Nakamata, Fumiya |
| contents | We present a model-based globally convergent policy gradient method (PGM) for linear quadratic Gaussian (LQG) control. Firstly, we establish equivalence between optimizing dynamic output feedback controllers and designing a static feedback gain for a system represented by a finite-length input-output history (IOH). This IOH-based approach allows us to explore LQG controllers within a parameter space defined by IOH gains. Secondly, by considering a control law comprising the IOH gain and a sufficiently small random perturbation, we show that the cost function, evaluated through the control law over IOH gains, is gradient-dominant and locally smooth, ensuring the global linear convergence of the PGM. Numerical simulations show that the dynamic controller learned by the proposed PGM is almost same as the LQG optimal controller, indicating promising results even in a reduced-order controller design. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_12173 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Globally Convergent Policy Gradient Method for Linear Quadratic Gaussian (LQG) Control Sadamoto, Tomonori Nakamata, Fumiya Optimization and Control Systems and Control We present a model-based globally convergent policy gradient method (PGM) for linear quadratic Gaussian (LQG) control. Firstly, we establish equivalence between optimizing dynamic output feedback controllers and designing a static feedback gain for a system represented by a finite-length input-output history (IOH). This IOH-based approach allows us to explore LQG controllers within a parameter space defined by IOH gains. Secondly, by considering a control law comprising the IOH gain and a sufficiently small random perturbation, we show that the cost function, evaluated through the control law over IOH gains, is gradient-dominant and locally smooth, ensuring the global linear convergence of the PGM. Numerical simulations show that the dynamic controller learned by the proposed PGM is almost same as the LQG optimal controller, indicating promising results even in a reduced-order controller design. |
| title | A Globally Convergent Policy Gradient Method for Linear Quadratic Gaussian (LQG) Control |
| topic | Optimization and Control Systems and Control |
| url | https://arxiv.org/abs/2312.12173 |