A-PSRO: A Unified Strategy Learning Method with Advantage Function for Normal-form Games
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916171434950656 |
|---|---|
| author | Hu, Yudong Li, Haoran Han, Congying Guo, Tiande Li, Mingqiang Li, Bonan |
| author_facet | Hu, Yudong Li, Haoran Han, Congying Guo, Tiande Li, Mingqiang Li, Bonan |
| contents | Solving Nash equilibrium is the key challenge in normal-form games with large strategy spaces, where open-ended learning frameworks offer an efficient approach. In this work, we propose an innovative unified open-ended learning framework A-PSRO, i.e., Advantage Policy Space Response Oracle, as a comprehensive framework for both zero-sum and general-sum games. In particular, we introduce the advantage function as an enhanced evaluation metric for strategies, enabling a unified learning objective for agents engaged in normal-form games. We prove that the advantage function exhibits favorable properties and is connected with the Nash equilibrium, which can be used as an objective to guide agents to learn strategies efficiently. Our experiments reveal that A-PSRO achieves a considerable decrease in exploitability in zero-sum games and an escalation in rewards in general-sum games, significantly outperforming previous PSRO algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_12520 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A-PSRO: A Unified Strategy Learning Method with Advantage Function for Normal-form Games Hu, Yudong Li, Haoran Han, Congying Guo, Tiande Li, Mingqiang Li, Bonan Computer Science and Game Theory Solving Nash equilibrium is the key challenge in normal-form games with large strategy spaces, where open-ended learning frameworks offer an efficient approach. In this work, we propose an innovative unified open-ended learning framework A-PSRO, i.e., Advantage Policy Space Response Oracle, as a comprehensive framework for both zero-sum and general-sum games. In particular, we introduce the advantage function as an enhanced evaluation metric for strategies, enabling a unified learning objective for agents engaged in normal-form games. We prove that the advantage function exhibits favorable properties and is connected with the Nash equilibrium, which can be used as an objective to guide agents to learn strategies efficiently. Our experiments reveal that A-PSRO achieves a considerable decrease in exploitability in zero-sum games and an escalation in rewards in general-sum games, significantly outperforming previous PSRO algorithms. |
| title | A-PSRO: A Unified Strategy Learning Method with Advantage Function for Normal-form Games |
| topic | Computer Science and Game Theory |
| url | https://arxiv.org/abs/2308.12520 |