PTDE: Personalized Training with Distilled Execution for Multi-Agent Reinforcement Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916214595387392 |
|---|---|
| author | Chen, Yiqun Mao, Hangyu Mao, Jiaxin Wu, Shiguang Zhang, Tianle Zhang, Bin Yang, Wei Chang, Hongxing |
| author_facet | Chen, Yiqun Mao, Hangyu Mao, Jiaxin Wu, Shiguang Zhang, Tianle Zhang, Bin Yang, Wei Chang, Hongxing |
| contents | Centralized Training with Decentralized Execution (CTDE) has emerged as a widely adopted paradigm in multi-agent reinforcement learning, emphasizing the utilization of global information for learning an enhanced joint $Q$-function or centralized critic. In contrast, our investigation delves into harnessing global information to directly enhance individual $Q$-functions or individual actors. Notably, we discover that applying identical global information universally across all agents proves insufficient for optimal performance. Consequently, we advocate for the customization of global information tailored to each agent, creating agent-personalized global information to bolster overall performance. Furthermore, we introduce a novel paradigm named Personalized Training with Distilled Execution (PTDE), wherein agent-personalized global information is distilled into the agent's local information. This distilled information is then utilized during decentralized execution, resulting in minimal performance degradation. PTDE can be seamlessly integrated with state-of-the-art algorithms, leading to notable performance enhancements across diverse benchmarks, including the SMAC benchmark, Google Research Football (GRF) benchmark, and Learning to Rank (LTR) task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_08872 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | PTDE: Personalized Training with Distilled Execution for Multi-Agent Reinforcement Learning Chen, Yiqun Mao, Hangyu Mao, Jiaxin Wu, Shiguang Zhang, Tianle Zhang, Bin Yang, Wei Chang, Hongxing Artificial Intelligence Machine Learning Multiagent Systems Centralized Training with Decentralized Execution (CTDE) has emerged as a widely adopted paradigm in multi-agent reinforcement learning, emphasizing the utilization of global information for learning an enhanced joint $Q$-function or centralized critic. In contrast, our investigation delves into harnessing global information to directly enhance individual $Q$-functions or individual actors. Notably, we discover that applying identical global information universally across all agents proves insufficient for optimal performance. Consequently, we advocate for the customization of global information tailored to each agent, creating agent-personalized global information to bolster overall performance. Furthermore, we introduce a novel paradigm named Personalized Training with Distilled Execution (PTDE), wherein agent-personalized global information is distilled into the agent's local information. This distilled information is then utilized during decentralized execution, resulting in minimal performance degradation. PTDE can be seamlessly integrated with state-of-the-art algorithms, leading to notable performance enhancements across diverse benchmarks, including the SMAC benchmark, Google Research Football (GRF) benchmark, and Learning to Rank (LTR) task. |
| title | PTDE: Personalized Training with Distilled Execution for Multi-Agent Reinforcement Learning |
| topic | Artificial Intelligence Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2210.08872 |