Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.02831 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916975335178240 |
|---|---|
| author | McKee, Kevin Alt, Eric Grebenisan, Andrew van Gelderen, Mick Miguel, Gary |
| author_facet | McKee, Kevin Alt, Eric Grebenisan, Andrew van Gelderen, Mick Miguel, Gary |
| contents | Exploration algorithms for reinforcement learning typically replace or augment the reward function with an additional ``intrinsic'' reward that trains the agent to seek previously unseen states of the environment. Here, we consider an exploration algorithm that exploits meta-learning, or learning to learn, such that the agent learns to maximize its exploration progress within a single episode, even between epochs of training. The agent learns a policy that aims to minimize the probability density of new observations with respect to all of its memories. In addition, it receives as feedback evaluations of the current observation density and retains that feedback in a recurrent network. By remembering trajectories of density, the agent learns to navigate a complex and growing landscape of familiarity in real-time, allowing it to maximize its exploration progress even in completely novel states of the environment for which its policy has not been trained. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_02831 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Meta-Learning to Explore via Memory Density Feedback McKee, Kevin Alt, Eric Grebenisan, Andrew van Gelderen, Mick Miguel, Gary Machine Learning Exploration algorithms for reinforcement learning typically replace or augment the reward function with an additional ``intrinsic'' reward that trains the agent to seek previously unseen states of the environment. Here, we consider an exploration algorithm that exploits meta-learning, or learning to learn, such that the agent learns to maximize its exploration progress within a single episode, even between epochs of training. The agent learns a policy that aims to minimize the probability density of new observations with respect to all of its memories. In addition, it receives as feedback evaluations of the current observation density and retains that feedback in a recurrent network. By remembering trajectories of density, the agent learns to navigate a complex and growing landscape of familiarity in real-time, allowing it to maximize its exploration progress even in completely novel states of the environment for which its policy has not been trained. |
| title | Meta-Learning to Explore via Memory Density Feedback |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.02831 |