Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908606939529216 |
|---|---|
| author | Byeon, Woohyeon Park, Giseung Chae, Jongseong Leshem, Amir Sung, Youngchul |
| author_facet | Byeon, Woohyeon Park, Giseung Chae, Jongseong Leshem, Amir Sung, Youngchul |
| contents | In this paper, we propose a provably convergent and practical framework for multi-objective reinforcement learning with max-min criterion. From a game-theoretic perspective, we reformulate max-min multi-objective reinforcement learning as a two-player zero-sum regularized continuous game and introduce an efficient algorithm based on mirror descent. Our approach simplifies the policy update while ensuring global last-iterate convergence. We provide a comprehensive theoretical analysis on our algorithm, including iteration complexity under both exact and approximate policy evaluations, as well as sample complexity bounds. To further enhance performance, we modify the proposed algorithm with adaptive regularization. Our experiments demonstrate the convergence behavior of the proposed algorithm in tabular settings, and our implementation for deep reinforcement learning significantly outperforms previous baselines in many MORL environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_20235 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach Byeon, Woohyeon Park, Giseung Chae, Jongseong Leshem, Amir Sung, Youngchul Machine Learning Artificial Intelligence In this paper, we propose a provably convergent and practical framework for multi-objective reinforcement learning with max-min criterion. From a game-theoretic perspective, we reformulate max-min multi-objective reinforcement learning as a two-player zero-sum regularized continuous game and introduce an efficient algorithm based on mirror descent. Our approach simplifies the policy update while ensuring global last-iterate convergence. We provide a comprehensive theoretical analysis on our algorithm, including iteration complexity under both exact and approximate policy evaluations, as well as sample complexity bounds. To further enhance performance, we modify the proposed algorithm with adaptive regularization. Our experiments demonstrate the convergence behavior of the proposed algorithm in tabular settings, and our implementation for deep reinforcement learning significantly outperforms previous baselines in many MORL environments. |
| title | Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.20235 |