Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929235581468672 |
|---|---|
| author | Bhardwaj, Arjun Rothfuss, Jonas Sukhija, Bhavya As, Yarden Hutter, Marco Coros, Stelian Krause, Andreas |
| author_facet | Bhardwaj, Arjun Rothfuss, Jonas Sukhija, Bhavya As, Yarden Hutter, Marco Coros, Stelian Krause, Andreas |
| contents | We introduce PACOH-RL, a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies to changing dynamics. PACOH-RL meta-learns priors for the dynamics model, allowing swift adaptation to new dynamics with minimal interaction data. Existing Meta-RL methods require abundant meta-learning data, limiting their applicability in settings such as robotics, where data is costly to obtain. To address this, PACOH-RL incorporates regularization and epistemic uncertainty quantification in both the meta-learning and task adaptation stages. When facing new dynamics, we use these uncertainty estimates to effectively guide exploration and data collection. Overall, this enables positive transfer, even when access to data from prior tasks or dynamic settings is severely limited. Our experiment results demonstrate that PACOH-RL outperforms model-based RL and model-based Meta-RL baselines in adapting to new dynamic conditions. Finally, on a real robotic car, we showcase the potential for efficient RL policy adaptation in diverse, data-scarce conditions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_07558 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning Bhardwaj, Arjun Rothfuss, Jonas Sukhija, Bhavya As, Yarden Hutter, Marco Coros, Stelian Krause, Andreas Machine Learning Robotics We introduce PACOH-RL, a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies to changing dynamics. PACOH-RL meta-learns priors for the dynamics model, allowing swift adaptation to new dynamics with minimal interaction data. Existing Meta-RL methods require abundant meta-learning data, limiting their applicability in settings such as robotics, where data is costly to obtain. To address this, PACOH-RL incorporates regularization and epistemic uncertainty quantification in both the meta-learning and task adaptation stages. When facing new dynamics, we use these uncertainty estimates to effectively guide exploration and data collection. Overall, this enables positive transfer, even when access to data from prior tasks or dynamic settings is severely limited. Our experiment results demonstrate that PACOH-RL outperforms model-based RL and model-based Meta-RL baselines in adapting to new dynamic conditions. Finally, on a real robotic car, we showcase the potential for efficient RL policy adaptation in diverse, data-scarce conditions. |
| title | Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2311.07558 |