Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.00381 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910768997335040 |
|---|---|
| author | Bajgar, Ondrej Gould, Sid William Mitta, Rohan Narayan Langford Liu, Jonathon Newcombe, Oliver Golden, Jack |
| author_facet | Bajgar, Ondrej Gould, Sid William Mitta, Rohan Narayan Langford Liu, Jonathon Newcombe, Oliver Golden, Jack |
| contents | As AI systems become increasingly autonomous, aligning their decision-making to human preferences is essential. In domains like autonomous driving or robotics, it is impossible to write down the reward function representing these preferences by hand. Inverse reinforcement learning (IRL) offers a promising approach to infer the unknown reward from demonstrations. However, obtaining human demonstrations can be costly. Active IRL addresses this challenge by strategically selecting the most informative scenarios for human demonstration, reducing the amount of required human effort. Where most prior work allowed querying the human for an action at one state at a time, we motivate and analyse scenarios where we collect longer trajectories. We provide an information-theoretic acquisition function, propose an efficient approximation scheme, and illustrate its performance through a set of gridworld experiments as groundwork for future work expanding to more general settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00381 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Toward Information Theoretic Active Inverse Reinforcement Learning Bajgar, Ondrej Gould, Sid William Mitta, Rohan Narayan Langford Liu, Jonathon Newcombe, Oliver Golden, Jack Machine Learning As AI systems become increasingly autonomous, aligning their decision-making to human preferences is essential. In domains like autonomous driving or robotics, it is impossible to write down the reward function representing these preferences by hand. Inverse reinforcement learning (IRL) offers a promising approach to infer the unknown reward from demonstrations. However, obtaining human demonstrations can be costly. Active IRL addresses this challenge by strategically selecting the most informative scenarios for human demonstration, reducing the amount of required human effort. Where most prior work allowed querying the human for an action at one state at a time, we motivate and analyse scenarios where we collect longer trajectories. We provide an information-theoretic acquisition function, propose an efficient approximation scheme, and illustrate its performance through a set of gridworld experiments as groundwork for future work expanding to more general settings. |
| title | Toward Information Theoretic Active Inverse Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.00381 |