Existence of $ε$-Nash Equilibria in Nonzero-Sum and Zero-Sum Markov Games with Standard Borel Spaces via Finite Model Approximations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918097858854912 |
|---|---|
| author | Saldi, Naci Arslan, Gurdal Yuksel, Serdar |
| author_facet | Saldi, Naci Arslan, Gurdal Yuksel, Serdar |
| contents | Establishing the existence of exact or near Markov or stationary perfect Nash equilibria in nonzero-sum Markov games over Borel spaces is a challenging problem with limited positive results. Motivated by problems in multi-agent and Bayesian learning, this paper demonstrates the existence of approximate Markov and stationary Nash equilibria for such games under mild regularity conditions. Our approach is constructive: For both compact and non-compact state spaces, we approximate the Borel model with finite state-action models and show that their equilibria correspond to \(ε\)-equilibria for the original game. Compared with previous results in the literature, which we comprehensively review, we provide more general and complementary conditions, along with explicit approximation models whose equilibria are $ε$-equilibria for the original model. For completeness, we also study the approximation of zero-sum Markov games and Markov teams to highlight the key differences between zero-sum and nonzero-sum settings. In particular, while for zero-sum and team games, joint weak (Feller) continuity of the transition kernel is sufficient (as the value function is continuous), this is not the case for general nonzero-sum games. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10805 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Existence of $ε$-Nash Equilibria in Nonzero-Sum and Zero-Sum Markov Games with Standard Borel Spaces via Finite Model Approximations Saldi, Naci Arslan, Gurdal Yuksel, Serdar Systems and Control Establishing the existence of exact or near Markov or stationary perfect Nash equilibria in nonzero-sum Markov games over Borel spaces is a challenging problem with limited positive results. Motivated by problems in multi-agent and Bayesian learning, this paper demonstrates the existence of approximate Markov and stationary Nash equilibria for such games under mild regularity conditions. Our approach is constructive: For both compact and non-compact state spaces, we approximate the Borel model with finite state-action models and show that their equilibria correspond to \(ε\)-equilibria for the original game. Compared with previous results in the literature, which we comprehensively review, we provide more general and complementary conditions, along with explicit approximation models whose equilibria are $ε$-equilibria for the original model. For completeness, we also study the approximation of zero-sum Markov games and Markov teams to highlight the key differences between zero-sum and nonzero-sum settings. In particular, while for zero-sum and team games, joint weak (Feller) continuity of the transition kernel is sufficient (as the value function is continuous), this is not the case for general nonzero-sum games. |
| title | Existence of $ε$-Nash Equilibria in Nonzero-Sum and Zero-Sum Markov Games with Standard Borel Spaces via Finite Model Approximations |
| topic | Systems and Control |
| url | https://arxiv.org/abs/2411.10805 |