Deep Reinforcement Learning with Task-Adaptive Retrieval via Hypernetwork
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911789817528320 |
|---|---|
| author | Jin, Yonggang Wang, Chenxu Zheng, Tianyu Xiang, Liuyu Yang, Yaodong Zhang, Junge Fu, Jie He, Zhaofeng |
| author_facet | Jin, Yonggang Wang, Chenxu Zheng, Tianyu Xiang, Liuyu Yang, Yaodong Zhang, Junge Fu, Jie He, Zhaofeng |
| contents | Deep reinforcement learning algorithms are usually impeded by sampling inefficiency, heavily depending on multiple interactions with the environment to acquire accurate decision-making capabilities. In contrast, humans rely on their hippocampus to retrieve relevant information from past experiences of relevant tasks, which guides their decision-making when learning a new task, rather than exclusively depending on environmental interactions. Nevertheless, designing a hippocampus-like module for an agent to incorporate past experiences into established reinforcement learning algorithms presents two challenges. The first challenge involves selecting the most relevant past experiences for the current task, and the second challenge is integrating such experiences into the decision network. To address these challenges, we propose a novel method that utilizes a retrieval network based on task-conditioned hypernetwork, which adapts the retrieval network's parameters depending on the task. At the same time, a dynamic modification mechanism enhances the collaborative efforts between the retrieval and decision networks. We evaluate the proposed method across various tasks within a multitask scenario in the Minigrid environment. The experimental results demonstrate that our proposed method significantly outperforms strong baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_10698 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Deep Reinforcement Learning with Task-Adaptive Retrieval via Hypernetwork Jin, Yonggang Wang, Chenxu Zheng, Tianyu Xiang, Liuyu Yang, Yaodong Zhang, Junge Fu, Jie He, Zhaofeng Machine Learning Artificial Intelligence Deep reinforcement learning algorithms are usually impeded by sampling inefficiency, heavily depending on multiple interactions with the environment to acquire accurate decision-making capabilities. In contrast, humans rely on their hippocampus to retrieve relevant information from past experiences of relevant tasks, which guides their decision-making when learning a new task, rather than exclusively depending on environmental interactions. Nevertheless, designing a hippocampus-like module for an agent to incorporate past experiences into established reinforcement learning algorithms presents two challenges. The first challenge involves selecting the most relevant past experiences for the current task, and the second challenge is integrating such experiences into the decision network. To address these challenges, we propose a novel method that utilizes a retrieval network based on task-conditioned hypernetwork, which adapts the retrieval network's parameters depending on the task. At the same time, a dynamic modification mechanism enhances the collaborative efforts between the retrieval and decision networks. We evaluate the proposed method across various tasks within a multitask scenario in the Minigrid environment. The experimental results demonstrate that our proposed method significantly outperforms strong baselines. |
| title | Deep Reinforcement Learning with Task-Adaptive Retrieval via Hypernetwork |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2306.10698 |