The Lagrangian Method for Solving Constrained Markov Games
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915196328476672 |
|---|---|
| author | Das, Soham Paternain, Santiago Chamon, Luiz F. O. Eksin, Ceyhun |
| author_facet | Das, Soham Paternain, Santiago Chamon, Luiz F. O. Eksin, Ceyhun |
| contents | We propose the concept of a Lagrangian game to solve constrained Markov games. Such games model scenarios where agents face cost constraints in addition to their individual rewards, that depend on both agent joint actions and the evolving environment state over time. Constrained Markov games form the formal mechanism behind safe multiagent reinforcement learning, providing a structured model for dynamic multiagent interactions in a multitude of settings, such as autonomous teams operating under local energy and time constraints, for example. We develop a primal-dual approach in which agents solve a Lagrangian game associated with the current Lagrange multiplier, simulate cost and reward trajectories over a fixed horizon, and update the multiplier using accrued experience. This update rule generates a new Lagrangian game, initiating the next iteration. Our key result consists in showing that the sequence of solutions to these Lagrangian games yields a nonstationary Nash solution for the original constrained Markov game. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_10561 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Lagrangian Method for Solving Constrained Markov Games Das, Soham Paternain, Santiago Chamon, Luiz F. O. Eksin, Ceyhun Optimization and Control Computer Science and Game Theory 93A16, 91A15 (Primary) 91A26, 49N15 (Secondary) I.2.11 We propose the concept of a Lagrangian game to solve constrained Markov games. Such games model scenarios where agents face cost constraints in addition to their individual rewards, that depend on both agent joint actions and the evolving environment state over time. Constrained Markov games form the formal mechanism behind safe multiagent reinforcement learning, providing a structured model for dynamic multiagent interactions in a multitude of settings, such as autonomous teams operating under local energy and time constraints, for example. We develop a primal-dual approach in which agents solve a Lagrangian game associated with the current Lagrange multiplier, simulate cost and reward trajectories over a fixed horizon, and update the multiplier using accrued experience. This update rule generates a new Lagrangian game, initiating the next iteration. Our key result consists in showing that the sequence of solutions to these Lagrangian games yields a nonstationary Nash solution for the original constrained Markov game. |
| title | The Lagrangian Method for Solving Constrained Markov Games |
| topic | Optimization and Control Computer Science and Game Theory 93A16, 91A15 (Primary) 91A26, 49N15 (Secondary) I.2.11 |
| url | https://arxiv.org/abs/2503.10561 |