Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.13224 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915069698244608 |
|---|---|
| author | Cao, Hongpeng Mao, Yanbing Sha, Lui Caccamo, Marco |
| author_facet | Cao, Hongpeng Mao, Yanbing Sha, Lui Caccamo, Marco |
| contents | Real-world accidents in learning-enabled CPS frequently occur in challenging corner cases. During the training of deep reinforcement learning (DRL) policy, the standard setup for training conditions is either fixed at a single initial condition or uniformly sampled from the admissible state space. This setup often overlooks the challenging but safety-critical corner cases. To bridge this gap, this paper proposes a physics-model-guided worst-case sampling strategy for training safe policies that can handle safety-critical cases toward guaranteed safety. Furthermore, we integrate the proposed worst-case sampling strategy into the physics-regulated deep reinforcement learning (Phy-DRL) framework to build a more data-efficient and safe learning algorithm for safety-critical CPS. We validate the proposed training strategy with Phy-DRL through extensive experiments on a simulated cart-pole system, a 2D quadrotor, a simulated and a real quadruped robot, showing remarkably improved sampling efficiency to learn more robust safe policies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13224 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning Cao, Hongpeng Mao, Yanbing Sha, Lui Caccamo, Marco Robotics Artificial Intelligence Machine Learning Real-world accidents in learning-enabled CPS frequently occur in challenging corner cases. During the training of deep reinforcement learning (DRL) policy, the standard setup for training conditions is either fixed at a single initial condition or uniformly sampled from the admissible state space. This setup often overlooks the challenging but safety-critical corner cases. To bridge this gap, this paper proposes a physics-model-guided worst-case sampling strategy for training safe policies that can handle safety-critical cases toward guaranteed safety. Furthermore, we integrate the proposed worst-case sampling strategy into the physics-regulated deep reinforcement learning (Phy-DRL) framework to build a more data-efficient and safe learning algorithm for safety-critical CPS. We validate the proposed training strategy with Phy-DRL through extensive experiments on a simulated cart-pole system, a 2D quadrotor, a simulated and a real quadruped robot, showing remarkably improved sampling efficiency to learn more robust safe policies. |
| title | Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2412.13224 |