Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915477462188032 |
|---|---|
| author | Wu, Zida Lauriere, Mathieu Geist, Matthieu Pietquin, Olivier Mehta, Ankur |
| author_facet | Wu, Zida Lauriere, Mathieu Geist, Matthieu Pietquin, Olivier Mehta, Ankur |
| contents | Mean Field Games (MFGs) offer a powerful framework for studying large-scale multi-agent systems. Yet, learning Nash equilibria in MFGs remains a challenging problem, particularly when the initial distribution is unknown or when the population is subject to common noise. In this paper, we introduce an efficient deep reinforcement learning (DRL) algorithm designed to achieve population-dependent Nash equilibria without relying on averaging or historical sampling, inspired by Munchausen RL and Online Mirror Descent. The resulting policy is adaptable to various initial distributions and sources of common noise. Through numerical experiments on seven canonical examples, we demonstrate that our algorithm exhibits superior convergence properties compared to state-of-the-art algorithms, particularly a DRL version of Fictitious Play for population-dependent policies. The performance in the presence of common noise underscores the robustness and adaptability of our approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_03030 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learning Wu, Zida Lauriere, Mathieu Geist, Matthieu Pietquin, Olivier Mehta, Ankur Machine Learning Multiagent Systems Robotics Systems and Control Mean Field Games (MFGs) offer a powerful framework for studying large-scale multi-agent systems. Yet, learning Nash equilibria in MFGs remains a challenging problem, particularly when the initial distribution is unknown or when the population is subject to common noise. In this paper, we introduce an efficient deep reinforcement learning (DRL) algorithm designed to achieve population-dependent Nash equilibria without relying on averaging or historical sampling, inspired by Munchausen RL and Online Mirror Descent. The resulting policy is adaptable to various initial distributions and sources of common noise. Through numerical experiments on seven canonical examples, we demonstrate that our algorithm exhibits superior convergence properties compared to state-of-the-art algorithms, particularly a DRL version of Fictitious Play for population-dependent policies. The performance in the presence of common noise underscores the robustness and adaptability of our approach. |
| title | Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learning |
| topic | Machine Learning Multiagent Systems Robotics Systems and Control |
| url | https://arxiv.org/abs/2509.03030 |