Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909898438082560 |
|---|---|
| author | Omidi, Saber Petrik, Marek Yoon, Se Young Begum, Momotaz |
| author_facet | Omidi, Saber Petrik, Marek Yoon, Se Young Begum, Momotaz |
| contents | Safety in stochastic control systems, which are subject to random noise with a known probability distribution, aims to compute policies that satisfy predefined operational constraints with high confidence throughout the uncertain evolution of the state variables. The unpredictable evolution of state variables poses a significant challenge for meeting predefined constraints using various control methods. To address this, we present a new algorithm that computes safe policies to determine the safety level across a finite state set. This algorithm reduces the safety objective to the standard average reward Markov Decision Process (MDP) objective. This reduction enables us to use standard techniques, such as linear programs, to compute and analyze safe policies. We validate the proposed method numerically on the Double Integrator and the Inverted Pendulum systems. Results indicate that the average-reward MDPs solution is more comprehensive, converges faster, and offers higher quality compared to the minimum discounted-reward solution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_08419 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs Omidi, Saber Petrik, Marek Yoon, Se Young Begum, Momotaz Systems and Control Machine Learning Robotics Optimization and Control Safety in stochastic control systems, which are subject to random noise with a known probability distribution, aims to compute policies that satisfy predefined operational constraints with high confidence throughout the uncertain evolution of the state variables. The unpredictable evolution of state variables poses a significant challenge for meeting predefined constraints using various control methods. To address this, we present a new algorithm that computes safe policies to determine the safety level across a finite state set. This algorithm reduces the safety objective to the standard average reward Markov Decision Process (MDP) objective. This reduction enables us to use standard techniques, such as linear programs, to compute and analyze safe policies. We validate the proposed method numerically on the Double Integrator and the Inverted Pendulum systems. Results indicate that the average-reward MDPs solution is more comprehensive, converges faster, and offers higher quality compared to the minimum discounted-reward solution. |
| title | Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs |
| topic | Systems and Control Machine Learning Robotics Optimization and Control |
| url | https://arxiv.org/abs/2511.08419 |