RL-GPT: Integrating Reinforcement Learning and Code-as-policy
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916142216380416 |
|---|---|
| author | Liu, Shaoteng Yuan, Haoqi Hu, Minda Li, Yanwei Chen, Yukang Liu, Shu Lu, Zongqing Jia, Jiaya |
| author_facet | Liu, Shaoteng Yuan, Haoqi Hu, Minda Li, Yanwei Chen, Yukang Liu, Shu Lu, Zongqing Jia, Jiaya |
| contents | Large Language Models (LLMs) have demonstrated proficiency in utilizing various tools by coding, yet they face limitations in handling intricate logic and precise control. In embodied tasks, high-level planning is amenable to direct coding, while low-level actions often necessitate task-specific refinement, such as Reinforcement Learning (RL). To seamlessly integrate both modalities, we introduce a two-level hierarchical framework, RL-GPT, comprising a slow agent and a fast agent. The slow agent analyzes actions suitable for coding, while the fast agent executes coding tasks. This decomposition effectively focuses each agent on specific tasks, proving highly efficient within our pipeline. Our approach outperforms traditional RL methods and existing GPT agents, demonstrating superior efficiency. In the Minecraft game, it rapidly obtains diamonds within a single day on an RTX3090. Additionally, it achieves SOTA performance across all designated MineDojo tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_19299 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RL-GPT: Integrating Reinforcement Learning and Code-as-policy Liu, Shaoteng Yuan, Haoqi Hu, Minda Li, Yanwei Chen, Yukang Liu, Shu Lu, Zongqing Jia, Jiaya Artificial Intelligence Machine Learning Large Language Models (LLMs) have demonstrated proficiency in utilizing various tools by coding, yet they face limitations in handling intricate logic and precise control. In embodied tasks, high-level planning is amenable to direct coding, while low-level actions often necessitate task-specific refinement, such as Reinforcement Learning (RL). To seamlessly integrate both modalities, we introduce a two-level hierarchical framework, RL-GPT, comprising a slow agent and a fast agent. The slow agent analyzes actions suitable for coding, while the fast agent executes coding tasks. This decomposition effectively focuses each agent on specific tasks, proving highly efficient within our pipeline. Our approach outperforms traditional RL methods and existing GPT agents, demonstrating superior efficiency. In the Minecraft game, it rapidly obtains diamonds within a single day on an RTX3090. Additionally, it achieves SOTA performance across all designated MineDojo tasks. |
| title | RL-GPT: Integrating Reinforcement Learning and Code-as-policy |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2402.19299 |