Learning Symbolic Task Decompositions for Multi-Agent Teams
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910835514802176 |
|---|---|
| author | Shah, Ameesh Lauffer, Niklas Chen, Thomas Pitta, Nikhil Seshia, Sanjit A. |
| author_facet | Shah, Ameesh Lauffer, Niklas Chen, Thomas Pitta, Nikhil Seshia, Sanjit A. |
| contents | One approach for improving sample efficiency in cooperative multi-agent learning is to decompose overall tasks into sub-tasks that can be assigned to individual agents. We study this problem in the context of reward machines: symbolic tasks that can be formally decomposed into sub-tasks. In order to handle settings without a priori knowledge of the environment, we introduce a framework that can learn the optimal decomposition from model-free interactions with the environment. Our method uses a task-conditioned architecture to simultaneously learn an optimal decomposition and the corresponding agents' policies for each sub-task. In doing so, we remove the need for a human to manually design the optimal decomposition while maintaining the sample-efficiency benefits of improved credit assignment. We provide experimental results in several deep reinforcement learning settings, demonstrating the efficacy of our approach. Our results indicate that our approach succeeds even in environments with codependent agent dynamics, enabling synchronous multi-agent learning not achievable in previous works. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_13376 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning Symbolic Task Decompositions for Multi-Agent Teams Shah, Ameesh Lauffer, Niklas Chen, Thomas Pitta, Nikhil Seshia, Sanjit A. Multiagent Systems Artificial Intelligence Machine Learning F.2.2 One approach for improving sample efficiency in cooperative multi-agent learning is to decompose overall tasks into sub-tasks that can be assigned to individual agents. We study this problem in the context of reward machines: symbolic tasks that can be formally decomposed into sub-tasks. In order to handle settings without a priori knowledge of the environment, we introduce a framework that can learn the optimal decomposition from model-free interactions with the environment. Our method uses a task-conditioned architecture to simultaneously learn an optimal decomposition and the corresponding agents' policies for each sub-task. In doing so, we remove the need for a human to manually design the optimal decomposition while maintaining the sample-efficiency benefits of improved credit assignment. We provide experimental results in several deep reinforcement learning settings, demonstrating the efficacy of our approach. Our results indicate that our approach succeeds even in environments with codependent agent dynamics, enabling synchronous multi-agent learning not achievable in previous works. |
| title | Learning Symbolic Task Decompositions for Multi-Agent Teams |
| topic | Multiagent Systems Artificial Intelligence Machine Learning F.2.2 |
| url | https://arxiv.org/abs/2502.13376 |