Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Games
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910007413440512 |
|---|---|
| author | Donmez, Ahmed Said Arslantas, Yuksel Sayin, Muhammed O. |
| author_facet | Donmez, Ahmed Said Arslantas, Yuksel Sayin, Muhammed O. |
| contents | We propose a novel independent and payoff-based learning framework for stochastic games that is model-free, game-agnostic, and gradient-free. The learning dynamics follow a best-response-type actor-critic architecture, where agents update their strategies (actors) using feedback from two distinct critics: a fast critic that intuitively responds to observed payoffs under limited information, and a slow critic that deliberatively approximates the solution to the underlying dynamic programming problem. Crucially, the learning process relies on non-equilibrium adaptation through smoothed best responses to observed payoffs. We establish convergence to (approximate) equilibria in two-agent zero-sum and multi-agent identical-interest stochastic games over an infinite horizon. This provides one of the first payoff-based and fully decentralized learning algorithms with theoretical guarantees in both settings. Empirical results further validate the robustness and effectiveness of the proposed approach across both classes of games. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00606 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Games Donmez, Ahmed Said Arslantas, Yuksel Sayin, Muhammed O. Machine Learning Computer Science and Game Theory We propose a novel independent and payoff-based learning framework for stochastic games that is model-free, game-agnostic, and gradient-free. The learning dynamics follow a best-response-type actor-critic architecture, where agents update their strategies (actors) using feedback from two distinct critics: a fast critic that intuitively responds to observed payoffs under limited information, and a slow critic that deliberatively approximates the solution to the underlying dynamic programming problem. Crucially, the learning process relies on non-equilibrium adaptation through smoothed best responses to observed payoffs. We establish convergence to (approximate) equilibria in two-agent zero-sum and multi-agent identical-interest stochastic games over an infinite horizon. This provides one of the first payoff-based and fully decentralized learning algorithms with theoretical guarantees in both settings. Empirical results further validate the robustness and effectiveness of the proposed approach across both classes of games. |
| title | Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Games |
| topic | Machine Learning Computer Science and Game Theory |
| url | https://arxiv.org/abs/2602.00606 |