Generalized Individual Q-learning for Polymatrix Games with Partial Observations
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914936284774400 |
|---|---|
| author | Donmez, Ahmed Said Sayin, Muhammed O. |
| author_facet | Donmez, Ahmed Said Sayin, Muhammed O. |
| contents | This paper addresses the challenge of limited observations in non-cooperative multi-agent systems where agents can have partial access to other agents' actions. We present the generalized individual Q-learning dynamics that combine belief-based and payoff-based learning for the networked interconnections of more than two self-interested agents. This approach leverages access to opponents' actions whenever possible, demonstrably achieving a faster (guaranteed) convergence to quantal response equilibrium in multi-agent zero-sum and potential polymatrix games. Notably, the dynamics reduce to the well-studied smoothed fictitious play and individual Q-learning under full and no access to opponent actions, respectively. We further quantify the improvement in convergence rate due to observing opponents' actions through numerical simulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_02663 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generalized Individual Q-learning for Polymatrix Games with Partial Observations Donmez, Ahmed Said Sayin, Muhammed O. Computer Science and Game Theory Systems and Control This paper addresses the challenge of limited observations in non-cooperative multi-agent systems where agents can have partial access to other agents' actions. We present the generalized individual Q-learning dynamics that combine belief-based and payoff-based learning for the networked interconnections of more than two self-interested agents. This approach leverages access to opponents' actions whenever possible, demonstrably achieving a faster (guaranteed) convergence to quantal response equilibrium in multi-agent zero-sum and potential polymatrix games. Notably, the dynamics reduce to the well-studied smoothed fictitious play and individual Q-learning under full and no access to opponent actions, respectively. We further quantify the improvement in convergence rate due to observing opponents' actions through numerical simulations. |
| title | Generalized Individual Q-learning for Polymatrix Games with Partial Observations |
| topic | Computer Science and Game Theory Systems and Control |
| url | https://arxiv.org/abs/2409.02663 |