Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929499404238848 |
|---|---|
| author | Zaman, Muhammad Aneeq uz Aggarwal, Shubham Bastopcu, Melih Başar, Tamer |
| author_facet | Zaman, Muhammad Aneeq uz Aggarwal, Shubham Bastopcu, Melih Başar, Tamer |
| contents | In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimization serves as a foundational approach for Reinforcement Learning (RL) techniques aimed at finding the NE, in this work we prove the linear convergence of a policy optimization algorithm which (subject to the adequacy of entropy regularization) is capable of provably attaining the NE. Furthermore, in scenarios where the entropy regularization proves insufficient, we present a $δ$-augmentation technique, which facilitates the achievement of an $ε$-NE within the game. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_00045 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games Zaman, Muhammad Aneeq uz Aggarwal, Shubham Bastopcu, Melih Başar, Tamer Computer Science and Game Theory Artificial Intelligence Machine Learning Multiagent Systems In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimization serves as a foundational approach for Reinforcement Learning (RL) techniques aimed at finding the NE, in this work we prove the linear convergence of a policy optimization algorithm which (subject to the adequacy of entropy regularization) is capable of provably attaining the NE. Furthermore, in scenarios where the entropy regularization proves insufficient, we present a $δ$-augmentation technique, which facilitates the achievement of an $ε$-NE within the game. |
| title | Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games |
| topic | Computer Science and Game Theory Artificial Intelligence Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2404.00045 |