Distributed Influence-Augmented Local Simulators for Parallel MARL in Large Networked Systems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914697812377600 |
|---|---|
| author | Suau, Miguel He, Jinke Çelikok, Mustafa Mert Spaan, Matthijs T. J. Oliehoek, Frans A. |
| author_facet | Suau, Miguel He, Jinke Çelikok, Mustafa Mert Spaan, Matthijs T. J. Oliehoek, Frans A. |
| contents | Due to its high sample complexity, simulation is, as of today, critical for the successful application of reinforcement learning. Many real-world problems, however, exhibit overly complex dynamics, which makes their full-scale simulation computationally slow. In this paper, we show how to decompose large networked systems of many agents into multiple local components such that we can build separate simulators that run independently and in parallel. To monitor the influence that the different local components exert on one another, each of these simulators is equipped with a learned model that is periodically trained on real trajectories. Our empirical results reveal that distributing the simulation among different processes not only makes it possible to train large multi-agent systems in just a few hours but also helps mitigate the negative effects of simultaneous learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2207_00288 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Distributed Influence-Augmented Local Simulators for Parallel MARL in Large Networked Systems Suau, Miguel He, Jinke Çelikok, Mustafa Mert Spaan, Matthijs T. J. Oliehoek, Frans A. Machine Learning Multiagent Systems Due to its high sample complexity, simulation is, as of today, critical for the successful application of reinforcement learning. Many real-world problems, however, exhibit overly complex dynamics, which makes their full-scale simulation computationally slow. In this paper, we show how to decompose large networked systems of many agents into multiple local components such that we can build separate simulators that run independently and in parallel. To monitor the influence that the different local components exert on one another, each of these simulators is equipped with a learned model that is periodically trained on real trajectories. Our empirical results reveal that distributing the simulation among different processes not only makes it possible to train large multi-agent systems in just a few hours but also helps mitigate the negative effects of simultaneous learning. |
| title | Distributed Influence-Augmented Local Simulators for Parallel MARL in Large Networked Systems |
| topic | Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2207.00288 |