Emergence of In-Context Reinforcement Learning from Noise Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913387208769536 |
|---|---|
| author | Zisman, Ilya Kurenkov, Vladislav Nikulin, Alexander Sinii, Viacheslav Kolesnikov, Sergey |
| author_facet | Zisman, Ilya Kurenkov, Vladislav Nikulin, Alexander Sinii, Viacheslav Kolesnikov, Sergey |
| contents | Recently, extensive studies in Reinforcement Learning have been carried out on the ability of transformers to adapt in-context to various environments and tasks. Current in-context RL methods are limited by their strict requirements for data, which needs to be generated by RL agents or labeled with actions from an optimal policy. In order to address this prevalent problem, we propose AD$^\varepsilon$, a new data acquisition approach that enables in-context Reinforcement Learning from noise-induced curriculum. We show that it is viable to construct a synthetic noise injection curriculum which helps to obtain learning histories. Moreover, we experimentally demonstrate that it is possible to alleviate the need for generation using optimal policies, with in-context RL still able to outperform the best suboptimal policy in a learning dataset by a 2x margin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_12275 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Emergence of In-Context Reinforcement Learning from Noise Distillation Zisman, Ilya Kurenkov, Vladislav Nikulin, Alexander Sinii, Viacheslav Kolesnikov, Sergey Machine Learning Recently, extensive studies in Reinforcement Learning have been carried out on the ability of transformers to adapt in-context to various environments and tasks. Current in-context RL methods are limited by their strict requirements for data, which needs to be generated by RL agents or labeled with actions from an optimal policy. In order to address this prevalent problem, we propose AD$^\varepsilon$, a new data acquisition approach that enables in-context Reinforcement Learning from noise-induced curriculum. We show that it is viable to construct a synthetic noise injection curriculum which helps to obtain learning histories. Moreover, we experimentally demonstrate that it is possible to alleviate the need for generation using optimal policies, with in-context RL still able to outperform the best suboptimal policy in a learning dataset by a 2x margin. |
| title | Emergence of In-Context Reinforcement Learning from Noise Distillation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2312.12275 |