Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.18924 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916303707570176 |
|---|---|
| author | Shu, Tianye Shang, Ke Gong, Cheng Nan, Yang Ishibuchi, Hisao |
| author_facet | Shu, Tianye Shang, Ke Gong, Cheng Nan, Yang Ishibuchi, Hisao |
| contents | For a control problem with multiple conflicting objectives, there exists a set of Pareto-optimal policies called the Pareto set instead of a single optimal policy. When a multi-objective control problem is continuous and complex, traditional multi-objective reinforcement learning (MORL) algorithms search for many Pareto-optimal deep policies to approximate the Pareto set, which is quite resource-consuming. In this paper, we propose a simple and resource-efficient MORL algorithm that learns a continuous representation of the Pareto set in a high-dimensional policy parameter space using a single hypernet. The learned hypernet can directly generate various well-trained policy networks for different user preferences. We compare our method with two state-of-the-art MORL algorithms on seven multi-objective continuous robot control problems. Experimental results show that our method achieves the best overall performance with the least training parameters. An interesting observation is that the Pareto set is well approximated by a curved line or surface in a high-dimensional parameter space. This observation will provide insight for researchers to design new MORL algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18924 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Pareto Set for Multi-Objective Continuous Robot Control Shu, Tianye Shang, Ke Gong, Cheng Nan, Yang Ishibuchi, Hisao Artificial Intelligence Machine Learning Robotics For a control problem with multiple conflicting objectives, there exists a set of Pareto-optimal policies called the Pareto set instead of a single optimal policy. When a multi-objective control problem is continuous and complex, traditional multi-objective reinforcement learning (MORL) algorithms search for many Pareto-optimal deep policies to approximate the Pareto set, which is quite resource-consuming. In this paper, we propose a simple and resource-efficient MORL algorithm that learns a continuous representation of the Pareto set in a high-dimensional policy parameter space using a single hypernet. The learned hypernet can directly generate various well-trained policy networks for different user preferences. We compare our method with two state-of-the-art MORL algorithms on seven multi-objective continuous robot control problems. Experimental results show that our method achieves the best overall performance with the least training parameters. An interesting observation is that the Pareto set is well approximated by a curved line or surface in a high-dimensional parameter space. This observation will provide insight for researchers to design new MORL algorithms. |
| title | Learning Pareto Set for Multi-Objective Continuous Robot Control |
| topic | Artificial Intelligence Machine Learning Robotics |
| url | https://arxiv.org/abs/2406.18924 |