Efficient Risk-sensitive Planning via Entropic Risk Measures
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912470521610240 |
|---|---|
| author | Marthe, Alexandre Bounan, Samuel Garivier, Aurélien Vernade, Claire |
| author_facet | Marthe, Alexandre Bounan, Samuel Garivier, Aurélien Vernade, Claire |
| contents | Risk-sensitive planning aims to identify policies maximizing some tail-focused metrics in Markov Decision Processes (MDPs). Such an optimization task can be very costly for the most widely used and interpretable metrics such as threshold probabilities or (Conditional) Values at Risk. Indeed, previous work showed that only Entropic Risk Measures (EntRM) can be efficiently optimized through dynamic programming, leaving a hard-to-interpret parameter to choose. We show that the computation of the full set of optimal policies for EntRM across parameter values leads to tight approximations for the metrics of interest. We prove that this optimality front can be computed effectively thanks to a novel structural analysis and smoothness properties of entropic risks. Empirical results demonstrate that our approach achieves strong performance in a variety of decision-making scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_20423 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Risk-sensitive Planning via Entropic Risk Measures Marthe, Alexandre Bounan, Samuel Garivier, Aurélien Vernade, Claire Machine Learning Artificial Intelligence Optimization and Control Probability Risk-sensitive planning aims to identify policies maximizing some tail-focused metrics in Markov Decision Processes (MDPs). Such an optimization task can be very costly for the most widely used and interpretable metrics such as threshold probabilities or (Conditional) Values at Risk. Indeed, previous work showed that only Entropic Risk Measures (EntRM) can be efficiently optimized through dynamic programming, leaving a hard-to-interpret parameter to choose. We show that the computation of the full set of optimal policies for EntRM across parameter values leads to tight approximations for the metrics of interest. We prove that this optimality front can be computed effectively thanks to a novel structural analysis and smoothness properties of entropic risks. Empirical results demonstrate that our approach achieves strong performance in a variety of decision-making scenarios. |
| title | Efficient Risk-sensitive Planning via Entropic Risk Measures |
| topic | Machine Learning Artificial Intelligence Optimization and Control Probability |
| url | https://arxiv.org/abs/2502.20423 |