Sable: a Performant, Efficient and Scalable Sequence Model for MARL
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909622700343296 |
|---|---|
| author | Mahjoub, Omayma Abramowitz, Sasha de Kock, Ruan Khlifi, Wiem Toit, Simon du Daniel, Jemma Nessir, Louay Ben Beyers, Louise Formanek, Claude Clark, Liam Pretorius, Arnu |
| author_facet | Mahjoub, Omayma Abramowitz, Sasha de Kock, Ruan Khlifi, Wiem Toit, Simon du Daniel, Jemma Nessir, Louay Ben Beyers, Louise Formanek, Claude Clark, Liam Pretorius, Arnu |
| contents | As multi-agent reinforcement learning (MARL) progresses towards solving larger and more complex problems, it becomes increasingly important that algorithms exhibit the key properties of (1) strong performance, (2) memory efficiency, and (3) scalability. In this work, we introduce Sable, a performant, memory-efficient, and scalable sequence modeling approach to MARL. Sable works by adapting the retention mechanism in Retentive Networks (Sun et al., 2023) to achieve computationally efficient processing of multi-agent observations with long context memory for temporal reasoning. Through extensive evaluations across six diverse environments, we demonstrate how Sable is able to significantly outperform existing state-of-the-art methods in a large number of diverse tasks (34 out of 45 tested). Furthermore, Sable maintains performance as we scale the number of agents, handling environments with more than a thousand agents while exhibiting a linear increase in memory usage. Finally, we conduct ablation studies to isolate the source of Sable's performance gains and confirm its efficient computational memory usage. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01706 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sable: a Performant, Efficient and Scalable Sequence Model for MARL Mahjoub, Omayma Abramowitz, Sasha de Kock, Ruan Khlifi, Wiem Toit, Simon du Daniel, Jemma Nessir, Louay Ben Beyers, Louise Formanek, Claude Clark, Liam Pretorius, Arnu Machine Learning Artificial Intelligence Multiagent Systems As multi-agent reinforcement learning (MARL) progresses towards solving larger and more complex problems, it becomes increasingly important that algorithms exhibit the key properties of (1) strong performance, (2) memory efficiency, and (3) scalability. In this work, we introduce Sable, a performant, memory-efficient, and scalable sequence modeling approach to MARL. Sable works by adapting the retention mechanism in Retentive Networks (Sun et al., 2023) to achieve computationally efficient processing of multi-agent observations with long context memory for temporal reasoning. Through extensive evaluations across six diverse environments, we demonstrate how Sable is able to significantly outperform existing state-of-the-art methods in a large number of diverse tasks (34 out of 45 tested). Furthermore, Sable maintains performance as we scale the number of agents, handling environments with more than a thousand agents while exhibiting a linear increase in memory usage. Finally, we conduct ablation studies to isolate the source of Sable's performance gains and confirm its efficient computational memory usage. |
| title | Sable: a Performant, Efficient and Scalable Sequence Model for MARL |
| topic | Machine Learning Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2410.01706 |