Power-Constrained Policy Gradient Methods for LQR
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916854188998656 |
|---|---|
| author | Verma, Ashwin Mitra, Aritra Ye, Lintao Gupta, Vijay |
| author_facet | Verma, Ashwin Mitra, Aritra Ye, Lintao Gupta, Vijay |
| contents | Consider a discrete-time Linear Quadratic Regulator (LQR) problem solved using policy gradient descent when the system matrices are unknown. The gradient is transmitted across a noisy channel over a finite time horizon using analog communication by a transmitter with an average power constraint. This is a simple setup at the intersection of reinforcement learning and networked control systems. We first consider a communication-constrained optimization framework, where gradient descent is applied to optimize a non-convex function under noisy gradient transmission. We provide an optimal power allocation algorithm that minimizes an upper bound on the expected optimality error at the final iteration and show that adaptive power allocation can lead to better convergence rate as compared to standard gradient descent with uniform power distribution. We then apply our results to the LQR setting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15806 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Power-Constrained Policy Gradient Methods for LQR Verma, Ashwin Mitra, Aritra Ye, Lintao Gupta, Vijay Optimization and Control Consider a discrete-time Linear Quadratic Regulator (LQR) problem solved using policy gradient descent when the system matrices are unknown. The gradient is transmitted across a noisy channel over a finite time horizon using analog communication by a transmitter with an average power constraint. This is a simple setup at the intersection of reinforcement learning and networked control systems. We first consider a communication-constrained optimization framework, where gradient descent is applied to optimize a non-convex function under noisy gradient transmission. We provide an optimal power allocation algorithm that minimizes an upper bound on the expected optimality error at the final iteration and show that adaptive power allocation can lead to better convergence rate as compared to standard gradient descent with uniform power distribution. We then apply our results to the LQR setting. |
| title | Power-Constrained Policy Gradient Methods for LQR |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2507.15806 |