Integrating LTL Constraints into PPO for Safe Reinforcement Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918364260073472 |
|---|---|
| author | Zhang, Maifang Yu, Hang Zuo, Qian Wang, Cheng Belle, Vaishak He, Fengxiang |
| author_facet | Zhang, Maifang Yu, Hang Zuo, Qian Wang, Cheng Belle, Vaishak He, Fengxiang |
| contents | This paper proposes Proximal Policy Optimization with Linear Temporal Logic Constraints (PPO-LTL), a framework that integrates safety constraints written in LTL into PPO for safe reinforcement learning. LTL constraints offer rigorous representations of complex safety requirements, such as regulations that broadly exist in robotics, enabling systematic monitoring of safety requirements. Violations against LTL constraints are monitored by limit-deterministic Büchi automata, and then translated by a logic-to-cost mechanism into penalty signals. The signals are further employed for guiding the policy optimization via the Lagrangian scheme. Extensive experiments on the Zones and CARLA environments show that our PPO-LTL can consistently reduce safety violations, while maintaining competitive performance, against the state-of-the-art methods. The code is at https://github.com/EVIEHub/PPO-LTL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01292 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Integrating LTL Constraints into PPO for Safe Reinforcement Learning Zhang, Maifang Yu, Hang Zuo, Qian Wang, Cheng Belle, Vaishak He, Fengxiang Machine Learning Artificial Intelligence Logic in Computer Science Robotics This paper proposes Proximal Policy Optimization with Linear Temporal Logic Constraints (PPO-LTL), a framework that integrates safety constraints written in LTL into PPO for safe reinforcement learning. LTL constraints offer rigorous representations of complex safety requirements, such as regulations that broadly exist in robotics, enabling systematic monitoring of safety requirements. Violations against LTL constraints are monitored by limit-deterministic Büchi automata, and then translated by a logic-to-cost mechanism into penalty signals. The signals are further employed for guiding the policy optimization via the Lagrangian scheme. Extensive experiments on the Zones and CARLA environments show that our PPO-LTL can consistently reduce safety violations, while maintaining competitive performance, against the state-of-the-art methods. The code is at https://github.com/EVIEHub/PPO-LTL. |
| title | Integrating LTL Constraints into PPO for Safe Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence Logic in Computer Science Robotics |
| url | https://arxiv.org/abs/2603.01292 |