Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915140527456256 |
|---|---|
| author | Röpke, Willem Reymond, Mathieu Mannion, Patrick Roijers, Diederik M. Nowé, Ann Rădulescu, Roxana |
| author_facet | Röpke, Willem Reymond, Mathieu Mannion, Patrick Roijers, Diederik M. Nowé, Ann Rădulescu, Roxana |
| contents | An important challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies to attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), which decomposes finding the Pareto front into a sequence of constrained single-objective problems. This enables us to guarantee convergence while providing an upper bound on the distance to undiscovered Pareto optimal solutions at each step. We evaluate IPRO using utility-based metrics and its hypervolume and find that it matches or outperforms methods that require additional assumptions. By leveraging problem-specific single-objective solvers, our approach also holds promise for applications beyond multi-objective reinforcement learning, such as planning and pathfinding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_07182 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning Röpke, Willem Reymond, Mathieu Mannion, Patrick Roijers, Diederik M. Nowé, Ann Rădulescu, Roxana Machine Learning An important challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies to attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), which decomposes finding the Pareto front into a sequence of constrained single-objective problems. This enables us to guarantee convergence while providing an upper bound on the distance to undiscovered Pareto optimal solutions at each step. We evaluate IPRO using utility-based metrics and its hypervolume and find that it matches or outperforms methods that require additional assumptions. By leveraging problem-specific single-objective solvers, our approach also holds promise for applications beyond multi-objective reinforcement learning, such as planning and pathfinding. |
| title | Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.07182 |