Decision-Focused Model-based Reinforcement Learning for Reward Transfer
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916489535160320 |
|---|---|
| author | Sharma, Abhishek Parbhoo, Sonali Gottesman, Omer Doshi-Velez, Finale |
| author_facet | Sharma, Abhishek Parbhoo, Sonali Gottesman, Omer Doshi-Velez, Finale |
| contents | Model-based reinforcement learning (MBRL) provides a way to learn a transition model of the environment, which can then be used to plan personalized policies for different patient cohorts and to understand the dynamics involved in the decision-making process. However, standard MBRL algorithms are either sensitive to changes in the reward function or achieve suboptimal performance on the task when the transition model is restricted. Motivated by the need to use simple and interpretable models in critical domains such as healthcare, we propose a novel robust decision-focused (RDF) algorithm that learns a transition model that achieves high returns while being robust to changes in the reward function. We demonstrate our RDF algorithm can be used with several model classes and planning algorithms. We also provide theoretical and empirical evidence, on a variety of simulators and real patient data, that RDF can learn simple yet effective models that can be used to plan personalized policies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_03365 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Decision-Focused Model-based Reinforcement Learning for Reward Transfer Sharma, Abhishek Parbhoo, Sonali Gottesman, Omer Doshi-Velez, Finale Machine Learning Artificial Intelligence Model-based reinforcement learning (MBRL) provides a way to learn a transition model of the environment, which can then be used to plan personalized policies for different patient cohorts and to understand the dynamics involved in the decision-making process. However, standard MBRL algorithms are either sensitive to changes in the reward function or achieve suboptimal performance on the task when the transition model is restricted. Motivated by the need to use simple and interpretable models in critical domains such as healthcare, we propose a novel robust decision-focused (RDF) algorithm that learns a transition model that achieves high returns while being robust to changes in the reward function. We demonstrate our RDF algorithm can be used with several model classes and planning algorithms. We also provide theoretical and empirical evidence, on a variety of simulators and real patient data, that RDF can learn simple yet effective models that can be used to plan personalized policies. |
| title | Decision-Focused Model-based Reinforcement Learning for Reward Transfer |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2304.03365 |