Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909179564785664 |
|---|---|
| author | Gerstgrasser, Matthias Danino, Tom Keren, Sarah |
| author_facet | Gerstgrasser, Matthias Danino, Tom Keren, Sarah |
| contents | We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training. The intuition behind this is that even a small number of relevant experiences from other agents could help each agent learn. Unlike many other multi-agent RL algorithms, this approach allows for largely decentralized training, requiring only a limited communication channel between agents. We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms. Further, sharing only a small number of highly relevant experiences outperforms sharing all experiences between agents, and the performance uplift from selective experience sharing is robust across a range of hyperparameters and DQN variants. A reference implementation of our algorithm is available at https://github.com/mgerstgrasser/super. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_00865 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning Gerstgrasser, Matthias Danino, Tom Keren, Sarah Machine Learning Artificial Intelligence Multiagent Systems Robotics We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training. The intuition behind this is that even a small number of relevant experiences from other agents could help each agent learn. Unlike many other multi-agent RL algorithms, this approach allows for largely decentralized training, requiring only a limited communication channel between agents. We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms. Further, sharing only a small number of highly relevant experiences outperforms sharing all experiences between agents, and the performance uplift from selective experience sharing is robust across a range of hyperparameters and DQN variants. A reference implementation of our algorithm is available at https://github.com/mgerstgrasser/super. |
| title | Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence Multiagent Systems Robotics |
| url | https://arxiv.org/abs/2311.00865 |