Ergodic-Risk Constrained Policy Optimization: The Linear Quadratic Case
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912265544925184 |
|---|---|
| author | Talebi, Shahriar Li, Na |
| author_facet | Talebi, Shahriar Li, Na |
| contents | Risk-sensitive control balances performance with resilience to unlikely events in uncertain systems. This paper introduces ergodic-risk criteria, which capture long-term cumulative risks through probabilistic limit theorems. By ensuring the dynamics exhibit strong ergodicity, we demonstrate that the time-correlated terms in these limiting criteria converge even with potentially heavy-tailed process noises as long as the noise has a finite fourth moment. Building upon this, we proposed the ergodic-risk constrained policy optimization which incorporates an ergodic-risk constraint to the classical Linear Quadratic Regulation (LQR) framework. We then propose a primal-dual policy optimization method that optimizes the average performance while satisfying the ergodic-risk constraints. Numerical results demonstrate that the new risk-constrained LQR not only optimizes average performance but also limits the asymptotic variance associated with the ergodic-risk criterion, making the closed-loop system more robust against sporadic large fluctuations in process noise. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05878 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Ergodic-Risk Constrained Policy Optimization: The Linear Quadratic Case Talebi, Shahriar Li, Na Optimization and Control Systems and Control Risk-sensitive control balances performance with resilience to unlikely events in uncertain systems. This paper introduces ergodic-risk criteria, which capture long-term cumulative risks through probabilistic limit theorems. By ensuring the dynamics exhibit strong ergodicity, we demonstrate that the time-correlated terms in these limiting criteria converge even with potentially heavy-tailed process noises as long as the noise has a finite fourth moment. Building upon this, we proposed the ergodic-risk constrained policy optimization which incorporates an ergodic-risk constraint to the classical Linear Quadratic Regulation (LQR) framework. We then propose a primal-dual policy optimization method that optimizes the average performance while satisfying the ergodic-risk constraints. Numerical results demonstrate that the new risk-constrained LQR not only optimizes average performance but also limits the asymptotic variance associated with the ergodic-risk criterion, making the closed-loop system more robust against sporadic large fluctuations in process noise. |
| title | Ergodic-Risk Constrained Policy Optimization: The Linear Quadratic Case |
| topic | Optimization and Control Systems and Control |
| url | https://arxiv.org/abs/2503.05878 |