Safe Exploration via Policy Priors
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918326490365952 |
|---|---|
| author | Wendl, Manuel As, Yarden Prajapat, Manish Pollak, Anton Coros, Stelian Krause, Andreas |
| author_facet | Wendl, Manuel As, Yarden Prajapat, Manish Pollak, Anton Coros, Stelian Krause, Andreas |
| contents | Safe exploration is a key requirement for reinforcement learning (RL) agents to learn and adapt online, beyond controlled (e.g. simulated) environments. In this work, we tackle this challenge by utilizing suboptimal yet conservative policies (e.g., obtained from offline data or simulators) as priors. Our approach, SOOPER, uses probabilistic dynamics models to optimistically explore, yet pessimistically fall back to the conservative policy prior if needed. We prove that SOOPER guarantees safety throughout learning, and establish convergence to an optimal policy by bounding its cumulative regret. Extensive experiments on key safe RL benchmarks and real-world hardware demonstrate that SOOPER is scalable, outperforms the state-of-the-art and validate our theoretical guarantees in practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19612 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Safe Exploration via Policy Priors Wendl, Manuel As, Yarden Prajapat, Manish Pollak, Anton Coros, Stelian Krause, Andreas Machine Learning Artificial Intelligence Robotics Safe exploration is a key requirement for reinforcement learning (RL) agents to learn and adapt online, beyond controlled (e.g. simulated) environments. In this work, we tackle this challenge by utilizing suboptimal yet conservative policies (e.g., obtained from offline data or simulators) as priors. Our approach, SOOPER, uses probabilistic dynamics models to optimistically explore, yet pessimistically fall back to the conservative policy prior if needed. We prove that SOOPER guarantees safety throughout learning, and establish convergence to an optimal policy by bounding its cumulative regret. Extensive experiments on key safe RL benchmarks and real-world hardware demonstrate that SOOPER is scalable, outperforms the state-of-the-art and validate our theoretical guarantees in practice. |
| title | Safe Exploration via Policy Priors |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2601.19612 |