Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929231135506432 |
|---|---|
| author | Goodall, Alexander W. Belardinelli, Francesco |
| author_facet | Goodall, Alexander W. Belardinelli, Francesco |
| contents | Shielding is a popular technique for achieving safe reinforcement learning (RL). However, classical shielding approaches come with quite restrictive assumptions making them difficult to deploy in complex environments, particularly those with continuous state or action spaces. In this paper we extend the more versatile approximate model-based shielding (AMBS) framework to the continuous setting. In particular we use Safety Gym as our test-bed, allowing for a more direct comparison of AMBS with popular constrained RL algorithms. We also provide strong probabilistic safety guarantees for the continuous setting. In addition, we propose two novel penalty techniques that directly modify the policy gradient, which empirically provide more stable convergence in our experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00816 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments Goodall, Alexander W. Belardinelli, Francesco Machine Learning Artificial Intelligence Shielding is a popular technique for achieving safe reinforcement learning (RL). However, classical shielding approaches come with quite restrictive assumptions making them difficult to deploy in complex environments, particularly those with continuous state or action spaces. In this paper we extend the more versatile approximate model-based shielding (AMBS) framework to the continuous setting. In particular we use Safety Gym as our test-bed, allowing for a more direct comparison of AMBS with popular constrained RL algorithms. We also provide strong probabilistic safety guarantees for the continuous setting. In addition, we propose two novel penalty techniques that directly modify the policy gradient, which empirically provide more stable convergence in our experiments. |
| title | Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.00816 |