FoX: Formation-aware exploration in multi-agent reinforcement learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914641136844800 |
|---|---|
| author | Jo, Yonghyeon Lee, Sunwoo Yeom, Junghyuk Han, Seungyul |
| author_facet | Jo, Yonghyeon Lee, Sunwoo Yeom, Junghyuk Han, Seungyul |
| contents | Recently, deep multi-agent reinforcement learning (MARL) has gained significant popularity due to its success in various cooperative multi-agent tasks. However, exploration still remains a challenging problem in MARL due to the partial observability of the agents and the exploration space that can grow exponentially as the number of agents increases. Firstly, in order to address the scalability issue of the exploration space, we define a formation-based equivalence relation on the exploration space and aim to reduce the search space by exploring only meaningful states in different formations. Then, we propose a novel formation-aware exploration (FoX) framework that encourages partially observable agents to visit the states in diverse formations by guiding them to be well aware of their current formation solely based on their own observations. Numerical results show that the proposed FoX framework significantly outperforms the state-of-the-art MARL algorithms on Google Research Football (GRF) and sparse Starcraft II multi-agent challenge (SMAC) tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_11272 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | FoX: Formation-aware exploration in multi-agent reinforcement learning Jo, Yonghyeon Lee, Sunwoo Yeom, Junghyuk Han, Seungyul Machine Learning Machine Learning (ML) - ML: Reinforcement Learning, Secondary Subject Areas: Multiagent Systems (MAS) - MAS: Multiagent Learning Recently, deep multi-agent reinforcement learning (MARL) has gained significant popularity due to its success in various cooperative multi-agent tasks. However, exploration still remains a challenging problem in MARL due to the partial observability of the agents and the exploration space that can grow exponentially as the number of agents increases. Firstly, in order to address the scalability issue of the exploration space, we define a formation-based equivalence relation on the exploration space and aim to reduce the search space by exploring only meaningful states in different formations. Then, we propose a novel formation-aware exploration (FoX) framework that encourages partially observable agents to visit the states in diverse formations by guiding them to be well aware of their current formation solely based on their own observations. Numerical results show that the proposed FoX framework significantly outperforms the state-of-the-art MARL algorithms on Google Research Football (GRF) and sparse Starcraft II multi-agent challenge (SMAC) tasks. |
| title | FoX: Formation-aware exploration in multi-agent reinforcement learning |
| topic | Machine Learning Machine Learning (ML) - ML: Reinforcement Learning, Secondary Subject Areas: Multiagent Systems (MAS) - MAS: Multiagent Learning |
| url | https://arxiv.org/abs/2308.11272 |