Federated reinforcement learning for robot motion planning with zero-shot generalization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929304846204928 |
|---|---|
| author | Yuan, Zhenyuan Xu, Siyuan Zhu, Minghui |
| author_facet | Yuan, Zhenyuan Xu, Siyuan Zhu, Minghui |
| contents | This paper considers the problem of learning a control policy for robot motion planning with zero-shot generalization, i.e., no data collection and policy adaptation is needed when the learned policy is deployed in new environments. We develop a federated reinforcement learning framework that enables collaborative learning of multiple learners and a central server, i.e., the Cloud, without sharing their raw data. In each iteration, each learner uploads its local control policy and the corresponding estimated normalized arrival time to the Cloud, which then computes the global optimum among the learners and broadcasts the optimal policy to the learners. Each learner then selects between its local control policy and that from the Cloud for next iteration. The proposed framework leverages on the derived zero-shot generalization guarantees on arrival time and safety. Theoretical guarantees on almost-sure convergence, almost consensus, Pareto improvement and optimality gap are also provided. Monte Carlo simulation is conducted to evaluate the proposed framework. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13245 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Federated reinforcement learning for robot motion planning with zero-shot generalization Yuan, Zhenyuan Xu, Siyuan Zhu, Minghui Systems and Control Artificial Intelligence Distributed, Parallel, and Cluster Computing Machine Learning Robotics This paper considers the problem of learning a control policy for robot motion planning with zero-shot generalization, i.e., no data collection and policy adaptation is needed when the learned policy is deployed in new environments. We develop a federated reinforcement learning framework that enables collaborative learning of multiple learners and a central server, i.e., the Cloud, without sharing their raw data. In each iteration, each learner uploads its local control policy and the corresponding estimated normalized arrival time to the Cloud, which then computes the global optimum among the learners and broadcasts the optimal policy to the learners. Each learner then selects between its local control policy and that from the Cloud for next iteration. The proposed framework leverages on the derived zero-shot generalization guarantees on arrival time and safety. Theoretical guarantees on almost-sure convergence, almost consensus, Pareto improvement and optimality gap are also provided. Monte Carlo simulation is conducted to evaluate the proposed framework. |
| title | Federated reinforcement learning for robot motion planning with zero-shot generalization |
| topic | Systems and Control Artificial Intelligence Distributed, Parallel, and Cluster Computing Machine Learning Robotics |
| url | https://arxiv.org/abs/2403.13245 |