Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2019
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914443664818176 |
|---|---|
| author | Bozkurt, Alper Kamil Wang, Yu Zavlanos, Michael M. Pajic, Miroslav |
| author_facet | Bozkurt, Alper Kamil Wang, Yu Zavlanos, Michael M. Pajic, Miroslav |
| contents | We present a reinforcement learning (RL) framework to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Process (MDP). Specifically, we learn a policy that maximizes the probability of satisfying the LTL formula without learning the transition probabilities. We introduce a novel rewarding and path-dependent discounting mechanism based on the LTL formula such that (i) an optimal policy maximizing the total discounted reward effectively maximizes the probabilities of satisfying LTL objectives, and (ii) a model-free RL algorithm using these rewards and discount factors is guaranteed to converge to such policy. Finally, we illustrate the applicability of our RL-based synthesis approach on two motion planning case studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1909_07299 |
| institution | arXiv |
| publishDate | 2019 |
| record_format | arxiv |
| spellingShingle | Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning Bozkurt, Alper Kamil Wang, Yu Zavlanos, Michael M. Pajic, Miroslav Robotics Artificial Intelligence Machine Learning We present a reinforcement learning (RL) framework to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Process (MDP). Specifically, we learn a policy that maximizes the probability of satisfying the LTL formula without learning the transition probabilities. We introduce a novel rewarding and path-dependent discounting mechanism based on the LTL formula such that (i) an optimal policy maximizing the total discounted reward effectively maximizes the probabilities of satisfying LTL objectives, and (ii) a model-free RL algorithm using these rewards and discount factors is guaranteed to converge to such policy. Finally, we illustrate the applicability of our RL-based synthesis approach on two motion planning case studies. |
| title | Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/1909.07299 |