Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913947484946432 |
|---|---|
| author | Ayabe, Shingo Otomo, Takuto Kera, Hiroshi Kawamoto, Kazuhiko |
| author_facet | Ayabe, Shingo Otomo, Takuto Kera, Hiroshi Kawamoto, Kazuhiko |
| contents | Offline reinforcement learning, which learns solely from datasets without environmental interaction, has gained attention. This approach, similar to traditional online deep reinforcement learning, is particularly promising for robot control applications. Nevertheless, its robustness against real-world challenges, such as joint actuator faults in robots, remains a critical concern. This study evaluates the robustness of existing offline reinforcement learning methods using legged robots from OpenAI Gym based on average episodic rewards. For robustness evaluation, we simulate failures by incorporating both random and adversarial perturbations, representing worst-case scenarios, into the joint torque signals. Our experiments show that existing offline reinforcement learning methods exhibit significant vulnerabilities to these action perturbations and are more vulnerable than online reinforcement learning methods, highlighting the need for more robust approaches in this field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_18781 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations Ayabe, Shingo Otomo, Takuto Kera, Hiroshi Kawamoto, Kazuhiko Robotics Machine Learning Offline reinforcement learning, which learns solely from datasets without environmental interaction, has gained attention. This approach, similar to traditional online deep reinforcement learning, is particularly promising for robot control applications. Nevertheless, its robustness against real-world challenges, such as joint actuator faults in robots, remains a critical concern. This study evaluates the robustness of existing offline reinforcement learning methods using legged robots from OpenAI Gym based on average episodic rewards. For robustness evaluation, we simulate failures by incorporating both random and adversarial perturbations, representing worst-case scenarios, into the joint torque signals. Our experiments show that existing offline reinforcement learning methods exhibit significant vulnerabilities to these action perturbations and are more vulnerable than online reinforcement learning methods, highlighting the need for more robust approaches in this field. |
| title | Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations |
| topic | Robotics Machine Learning |
| url | https://arxiv.org/abs/2412.18781 |