Safe Exploration Using Bayesian World Models and Log-Barrier Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914789466308608 |
|---|---|
| author | As, Yarden Sukhija, Bhavya Krause, Andreas |
| author_facet | As, Yarden Sukhija, Bhavya Krause, Andreas |
| contents | A major challenge in deploying reinforcement learning in online tasks is ensuring that safety is maintained throughout the learning process. In this work, we propose CERL, a new method for solving constrained Markov decision processes while keeping the policy safe during learning. Our method leverages Bayesian world models and suggests policies that are pessimistic w.r.t. the model's epistemic uncertainty. This makes CERL robust towards model inaccuracies and leads to safe exploration during learning. In our experiments, we demonstrate that CERL outperforms the current state-of-the-art in terms of safety and optimality in solving CMDPs from image observations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05890 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Safe Exploration Using Bayesian World Models and Log-Barrier Optimization As, Yarden Sukhija, Bhavya Krause, Andreas Machine Learning Artificial Intelligence A major challenge in deploying reinforcement learning in online tasks is ensuring that safety is maintained throughout the learning process. In this work, we propose CERL, a new method for solving constrained Markov decision processes while keeping the policy safe during learning. Our method leverages Bayesian world models and suggests policies that are pessimistic w.r.t. the model's epistemic uncertainty. This makes CERL robust towards model inaccuracies and leads to safe exploration during learning. In our experiments, we demonstrate that CERL outperforms the current state-of-the-art in terms of safety and optimality in solving CMDPs from image observations. |
| title | Safe Exploration Using Bayesian World Models and Log-Barrier Optimization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2405.05890 |