Inverse Reinforcement Learning with Multiple Planning Horizons
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913520320249856 |
|---|---|
| author | Yao, Jiayu Pan, Weiwei Doshi-Velez, Finale Engelhardt, Barbara E |
| author_facet | Yao, Jiayu Pan, Weiwei Doshi-Velez, Finale Engelhardt, Barbara E |
| contents | In this work, we study an inverse reinforcement learning (IRL) problem where the experts are planning under a shared reward function but with different, unknown planning horizons. Without the knowledge of discount factors, the reward function has a larger feasible solution set, which makes it harder for existing IRL approaches to identify a reward function. To overcome this challenge, we develop algorithms that can learn a global multi-agent reward function with agent-specific discount factors that reconstruct the expert policies. We characterize the feasible solution space of the reward function and discount factors for both algorithms and demonstrate the generalizability of the learned reward function across multiple domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_18051 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Inverse Reinforcement Learning with Multiple Planning Horizons Yao, Jiayu Pan, Weiwei Doshi-Velez, Finale Engelhardt, Barbara E Machine Learning In this work, we study an inverse reinforcement learning (IRL) problem where the experts are planning under a shared reward function but with different, unknown planning horizons. Without the knowledge of discount factors, the reward function has a larger feasible solution set, which makes it harder for existing IRL approaches to identify a reward function. To overcome this challenge, we develop algorithms that can learn a global multi-agent reward function with agent-specific discount factors that reconstruct the expert policies. We characterize the feasible solution space of the reward function and discount factors for both algorithms and demonstrate the generalizability of the learned reward function across multiple domains. |
| title | Inverse Reinforcement Learning with Multiple Planning Horizons |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2409.18051 |