Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911811584917504 |
|---|---|
| author | Mazumdar, Abhijit Wisniewski, Rafal Bujorianu, Manuela L. |
| author_facet | Mazumdar, Abhijit Wisniewski, Rafal Bujorianu, Manuela L. |
| contents | In this paper, we present an online reinforcement learning algorithm for constrained Markov decision processes with a safety constraint. Despite the necessary attention of the scientific community, considering stochastic stopping time, the problem of learning optimal policy without violating safety constraints during the learning phase is yet to be addressed. To this end, we propose an algorithm based on linear programming that does not require a process model. We show that the learned policy is safe with high confidence. We also propose a method to compute a safe baseline policy, which is central in developing algorithms that do not violate the safety constraints. Finally, we provide simulation results to show the efficacy of the proposed algorithm. Further, we demonstrate that efficient exploration can be achieved by defining a subset of the state-space called proxy set. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_15928 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time Mazumdar, Abhijit Wisniewski, Rafal Bujorianu, Manuela L. Machine Learning Optimization and Control In this paper, we present an online reinforcement learning algorithm for constrained Markov decision processes with a safety constraint. Despite the necessary attention of the scientific community, considering stochastic stopping time, the problem of learning optimal policy without violating safety constraints during the learning phase is yet to be addressed. To this end, we propose an algorithm based on linear programming that does not require a process model. We show that the learned policy is safe with high confidence. We also propose a method to compute a safe baseline policy, which is central in developing algorithms that do not violate the safety constraints. Finally, we provide simulation results to show the efficacy of the proposed algorithm. Further, we demonstrate that efficient exploration can be achieved by defining a subset of the state-space called proxy set. |
| title | Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2403.15928 |