Learning the Target Network in Function Space
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910615545577472 |
|---|---|
| author | Asadi, Kavosh Liu, Yao Sabach, Shoham Yin, Ming Fakoor, Rasool |
| author_facet | Asadi, Kavosh Liu, Yao Sabach, Shoham Yin, Ming Fakoor, Rasool |
| contents | We focus on the task of learning the value function in the reinforcement learning (RL) setting. This task is often solved by updating a pair of online and target networks while ensuring that the parameters of these two networks are equivalent. We propose Lookahead-Replicate (LR), a new value-function approximation algorithm that is agnostic to this parameter-space equivalence. Instead, the LR algorithm is designed to maintain an equivalence between the two networks in the function space. This value-based equivalence is obtained by employing a new target-network update. We show that LR leads to a convergent behavior in learning the value function. We also present empirical results demonstrating that LR-based target-network updates significantly improve deep RL on the Atari benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01838 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning the Target Network in Function Space Asadi, Kavosh Liu, Yao Sabach, Shoham Yin, Ming Fakoor, Rasool Machine Learning Artificial Intelligence We focus on the task of learning the value function in the reinforcement learning (RL) setting. This task is often solved by updating a pair of online and target networks while ensuring that the parameters of these two networks are equivalent. We propose Lookahead-Replicate (LR), a new value-function approximation algorithm that is agnostic to this parameter-space equivalence. Instead, the LR algorithm is designed to maintain an equivalence between the two networks in the function space. This value-based equivalence is obtained by employing a new target-network update. We show that LR leads to a convergent behavior in learning the value function. We also present empirical results demonstrating that LR-based target-network updates significantly improve deep RL on the Atari benchmark. |
| title | Learning the Target Network in Function Space |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.01838 |