Counterfactual Strategies for Markov Decision Processes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915287273570304 |
|---|---|
| author | Kobialka, Paul Gerlach, Lina Leofante, Francesco Ábrahám, Erika Tarifa, Silvia Lizeth Tapia Johnsen, Einar Broch |
| author_facet | Kobialka, Paul Gerlach, Lina Leofante, Francesco Ábrahám, Erika Tarifa, Silvia Lizeth Tapia Johnsen, Einar Broch |
| contents | Counterfactuals are widely used in AI to explain how minimal changes to a model's input can lead to a different output. However, established methods for computing counterfactuals typically focus on one-step decision-making, and are not directly applicable to sequential decision-making tasks. This paper fills this gap by introducing counterfactual strategies for Markov Decision Processes (MDPs). During MDP execution, a strategy decides which of the enabled actions (with known probabilistic effects) to execute next. Given an initial strategy that reaches an undesired outcome with a probability above some limit, we identify minimal changes to the initial strategy to reduce that probability below the limit. We encode such counterfactual strategies as solutions to non-linear optimization problems, and further extend our encoding to synthesize diverse counterfactual strategies. We evaluate our approach on four real-world datasets and demonstrate its practical viability in sophisticated sequential decision-making tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_09412 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Counterfactual Strategies for Markov Decision Processes Kobialka, Paul Gerlach, Lina Leofante, Francesco Ábrahám, Erika Tarifa, Silvia Lizeth Tapia Johnsen, Einar Broch Artificial Intelligence I.2.m Counterfactuals are widely used in AI to explain how minimal changes to a model's input can lead to a different output. However, established methods for computing counterfactuals typically focus on one-step decision-making, and are not directly applicable to sequential decision-making tasks. This paper fills this gap by introducing counterfactual strategies for Markov Decision Processes (MDPs). During MDP execution, a strategy decides which of the enabled actions (with known probabilistic effects) to execute next. Given an initial strategy that reaches an undesired outcome with a probability above some limit, we identify minimal changes to the initial strategy to reduce that probability below the limit. We encode such counterfactual strategies as solutions to non-linear optimization problems, and further extend our encoding to synthesize diverse counterfactual strategies. We evaluate our approach on four real-world datasets and demonstrate its practical viability in sophisticated sequential decision-making tasks. |
| title | Counterfactual Strategies for Markov Decision Processes |
| topic | Artificial Intelligence I.2.m |
| url | https://arxiv.org/abs/2505.09412 |