GRAM: Generalization in Deep RL with a Robust Adaptation Module
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908672212336640 |
|---|---|
| author | Queeney, James Cai, Xiaoyi Schperberg, Alexander Corcodel, Radu Benosman, Mouhacine How, Jonathan P. |
| author_facet | Queeney, James Cai, Xiaoyi Schperberg, Alexander Corcodel, Radu Benosman, Mouhacine How, Jonathan P. |
| contents | The reliable deployment of deep reinforcement learning in real-world settings requires the ability to generalize across a variety of conditions, including both in-distribution scenarios seen during training as well as novel out-of-distribution scenarios. In this work, we present a framework for dynamics generalization in deep reinforcement learning that unifies these two distinct types of generalization within a single architecture. We introduce a robust adaptation module that provides a mechanism for identifying and reacting to both in-distribution and out-of-distribution environment dynamics, along with a joint training pipeline that combines the goals of in-distribution adaptation and out-of-distribution robustness. Our algorithm GRAM achieves strong generalization performance across in-distribution and out-of-distribution scenarios upon deployment, which we demonstrate through extensive simulation and hardware locomotion experiments on a quadruped robot. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04323 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GRAM: Generalization in Deep RL with a Robust Adaptation Module Queeney, James Cai, Xiaoyi Schperberg, Alexander Corcodel, Radu Benosman, Mouhacine How, Jonathan P. Machine Learning Artificial Intelligence Robotics The reliable deployment of deep reinforcement learning in real-world settings requires the ability to generalize across a variety of conditions, including both in-distribution scenarios seen during training as well as novel out-of-distribution scenarios. In this work, we present a framework for dynamics generalization in deep reinforcement learning that unifies these two distinct types of generalization within a single architecture. We introduce a robust adaptation module that provides a mechanism for identifying and reacting to both in-distribution and out-of-distribution environment dynamics, along with a joint training pipeline that combines the goals of in-distribution adaptation and out-of-distribution robustness. Our algorithm GRAM achieves strong generalization performance across in-distribution and out-of-distribution scenarios upon deployment, which we demonstrate through extensive simulation and hardware locomotion experiments on a quadruped robot. |
| title | GRAM: Generalization in Deep RL with a Robust Adaptation Module |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2412.04323 |