Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908446838751232 |
|---|---|
| author | Chang, Ting-Jui Shahrampour, Shahin |
| author_facet | Chang, Ting-Jui Shahrampour, Shahin |
| contents | Recent advancement in online optimization and control has provided novel tools to study online linear quadratic regulator (LQR) problems, where cost matrices are time-varying and unknown in advance. In this work, we study the online linear quadratic Gaussian (LQG) problem over the manifold of stabilizing controllers that are linearly constrained to impose physical conditions such as sparsity. By adopting a Riemannian perspective, we propose the online Newton on manifold (ONM) algorithm, which generates an online controller on-the-fly based on the second-order information of the cost function sequence. To quantify the algorithm performance, we use the notion of regret, defined as the sub-optimality of the algorithm cumulative cost against a (locally) minimizing controller sequence. We establish a regret bound in terms of the path-length of the benchmark minimizer sequence, and we further verify the effectiveness of ONM via simulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_08553 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG Chang, Ting-Jui Shahrampour, Shahin Optimization and Control Machine Learning Systems and Control Recent advancement in online optimization and control has provided novel tools to study online linear quadratic regulator (LQR) problems, where cost matrices are time-varying and unknown in advance. In this work, we study the online linear quadratic Gaussian (LQG) problem over the manifold of stabilizing controllers that are linearly constrained to impose physical conditions such as sparsity. By adopting a Riemannian perspective, we propose the online Newton on manifold (ONM) algorithm, which generates an online controller on-the-fly based on the second-order information of the cost function sequence. To quantify the algorithm performance, we use the notion of regret, defined as the sub-optimality of the algorithm cumulative cost against a (locally) minimizing controller sequence. We establish a regret bound in terms of the path-length of the benchmark minimizer sequence, and we further verify the effectiveness of ONM via simulations. |
| title | Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG |
| topic | Optimization and Control Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2403.08553 |