Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929391413493760 |
|---|---|
| author | Kalagarla, Krishna C. Kartik, Dhruva Shen, Dongming Jain, Rahul Nayyar, Ashutosh Nuzzo, Pierluigi |
| author_facet | Kalagarla, Krishna C. Kartik, Dhruva Shen, Dongming Jain, Rahul Nayyar, Ashutosh Nuzzo, Pierluigi |
| contents | Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially observable. Moreover, it could encompass a team of multiple agents with a common objective but disparate information structures and constraints. In this paper, we first introduce an optimal control theory for partially observable Markov decision processes (POMDPs) with finite linear temporal logic constraints. We provide a structured methodology for synthesizing policies that maximize a cumulative reward while ensuring that the probability of satisfying a temporal logic constraint is sufficiently high. Our approach comes with guarantees on approximate reward optimality and constraint satisfaction. We then build on this approach to design an optimal control framework for logically constrained multi-agent settings with information asymmetry. We illustrate the effectiveness of our approach by implementing it on several case studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_14736 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes Kalagarla, Krishna C. Kartik, Dhruva Shen, Dongming Jain, Rahul Nayyar, Ashutosh Nuzzo, Pierluigi Artificial Intelligence Formal Languages and Automata Theory Systems and Control Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially observable. Moreover, it could encompass a team of multiple agents with a common objective but disparate information structures and constraints. In this paper, we first introduce an optimal control theory for partially observable Markov decision processes (POMDPs) with finite linear temporal logic constraints. We provide a structured methodology for synthesizing policies that maximize a cumulative reward while ensuring that the probability of satisfying a temporal logic constraint is sufficiently high. Our approach comes with guarantees on approximate reward optimality and constraint satisfaction. We then build on this approach to design an optimal control framework for logically constrained multi-agent settings with information asymmetry. We illustrate the effectiveness of our approach by implementing it on several case studies. |
| title | Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes |
| topic | Artificial Intelligence Formal Languages and Automata Theory Systems and Control |
| url | https://arxiv.org/abs/2305.14736 |