Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.04120 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912323754524672 |
|---|---|
| author | Cleynen, Alice de Saporta, Benoîte Rossini, Orlane Sabbadin, Régis Vernay, Amélie |
| author_facet | Cleynen, Alice de Saporta, Benoîte Rossini, Orlane Sabbadin, Régis Vernay, Amélie |
| contents | Control theory plays a pivotal role in understanding and optimizing the behavior of complex dynamical systems across various scientific and engineering disciplines. Two key frameworks that have emerged for modeling and solving control problems in stochastic systems are piecewise deterministic Markov processes (PDMPs) and Markov decision processes (MDPs). Each framework has its unique strengths, and their intersection offers promising opportunities for tackling a broad class of problems, particularly in the context of impulse controls and decision-making in complex systems.
The relationship between PDMPs and MDPs is a natural subject of exploration, as embedding impulse control problems for PDMPs into the MDP framework could open new avenues for their analysis and resolution. Specifically, this integration would allow leveraging the computational and theoretical tools developed for MDPs to address the challenges inherent in PDMPs. On the other hand, PDMPs can offer a versatile and simple paradigm to model continuous time problems that are often described as discrete-time MDPs parametrized by complex transition kernels. This transformation has the potential to bridge the gap between the two frameworks, enabling solutions to previously intractable problems and expanding the scope of both fields. This paper presents a comprehensive review of two research domains, illustrated through a recurring medical example. The example is revisited and progressively formalized within the framework of thevarious concepts and objects introduced |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04120 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bridging Impulse Control of Piecewise Deterministic Markov Processes and Markov Decision Processes: Frameworks, Extensions, and Open Challenges Cleynen, Alice de Saporta, Benoîte Rossini, Orlane Sabbadin, Régis Vernay, Amélie Methodology Systems and Control Control theory plays a pivotal role in understanding and optimizing the behavior of complex dynamical systems across various scientific and engineering disciplines. Two key frameworks that have emerged for modeling and solving control problems in stochastic systems are piecewise deterministic Markov processes (PDMPs) and Markov decision processes (MDPs). Each framework has its unique strengths, and their intersection offers promising opportunities for tackling a broad class of problems, particularly in the context of impulse controls and decision-making in complex systems. The relationship between PDMPs and MDPs is a natural subject of exploration, as embedding impulse control problems for PDMPs into the MDP framework could open new avenues for their analysis and resolution. Specifically, this integration would allow leveraging the computational and theoretical tools developed for MDPs to address the challenges inherent in PDMPs. On the other hand, PDMPs can offer a versatile and simple paradigm to model continuous time problems that are often described as discrete-time MDPs parametrized by complex transition kernels. This transformation has the potential to bridge the gap between the two frameworks, enabling solutions to previously intractable problems and expanding the scope of both fields. This paper presents a comprehensive review of two research domains, illustrated through a recurring medical example. The example is revisited and progressively formalized within the framework of thevarious concepts and objects introduced |
| title | Bridging Impulse Control of Piecewise Deterministic Markov Processes and Markov Decision Processes: Frameworks, Extensions, and Open Challenges |
| topic | Methodology Systems and Control |
| url | https://arxiv.org/abs/2501.04120 |