Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908330120708096 |
|---|---|
| author | Wang, Jun Hasanbeig, Hosein Tan, Kaiyuan Sun, Zihe Kantaros, Yiannis |
| author_facet | Wang, Jun Hasanbeig, Hosein Tan, Kaiyuan Sun, Zihe Kantaros, Yiannis |
| contents | This paper addresses the problem of designing control policies for agents with unknown stochastic dynamics and control objectives specified using Linear Temporal Logic (LTL). Recent Deep Reinforcement Learning (DRL) algorithms have aimed to compute policies that maximize the satisfaction probability of LTL formulas, but they often suffer from slow learning performance. To address this, we introduce a novel Deep Q-learning algorithm that significantly improves learning speed. The enhanced sample efficiency stems from a mission-driven exploration strategy that prioritizes exploration towards directions likely to contribute to mission success. Identifying these directions relies on an automaton representation of the LTL task as well as a learned neural network that partially models the agent-environment interaction. We provide comparative experiments demonstrating the efficiency of our algorithm on robot navigation tasks in unseen environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_17059 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications Wang, Jun Hasanbeig, Hosein Tan, Kaiyuan Sun, Zihe Kantaros, Yiannis Robotics Artificial Intelligence Machine Learning This paper addresses the problem of designing control policies for agents with unknown stochastic dynamics and control objectives specified using Linear Temporal Logic (LTL). Recent Deep Reinforcement Learning (DRL) algorithms have aimed to compute policies that maximize the satisfaction probability of LTL formulas, but they often suffer from slow learning performance. To address this, we introduce a novel Deep Q-learning algorithm that significantly improves learning speed. The enhanced sample efficiency stems from a mission-driven exploration strategy that prioritizes exploration towards directions likely to contribute to mission success. Identifying these directions relies on an automaton representation of the LTL task as well as a learned neural network that partially models the agent-environment interaction. We provide comparative experiments demonstrating the efficiency of our algorithm on robot navigation tasks in unseen environments. |
| title | Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2311.17059 |