Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909593851920384 |
|---|---|
| author | Chen, Jiayi Wang, Shuai Li, Guoliang Xu, Wei Zhu, Guangxu Ng, Derrick Wing Kwan Xu, Chengzhong |
| author_facet | Chen, Jiayi Wang, Shuai Li, Guoliang Xu, Wei Zhu, Guangxu Ng, Derrick Wing Kwan Xu, Chengzhong |
| contents | Navigating autonomous vehicles in open scenarios is a challenge due to the difficulties in handling unseen objects. Existing solutions either rely on small models that struggle with generalization or large models that are resource-intensive. While collaboration between the two offers a promising solution, the key challenge is deciding when and how to engage the large model. To address this issue, this paper proposes opportunistic collaborative planning (OCP), which seamlessly integrates efficient local models with powerful cloud models through two key innovations. First, we propose large vision model guided model predictive control (LVM-MPC), which leverages the cloud for LVM perception and decision making. The cloud output serves as a global guidance for a local MPC, thereby forming a closed-loop perception-to-control system. Second, to determine the best timing for large model query and service, we propose collaboration timing optimization (CTO), including object detection confidence thresholding (ODCT) and cloud forward simulation (CFS), to decide when to seek cloud assistance and when to offer cloud service. Extensive experiments show that the proposed OCP outperforms existing methods in terms of both navigation time and success rate. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18057 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization Chen, Jiayi Wang, Shuai Li, Guoliang Xu, Wei Zhu, Guangxu Ng, Derrick Wing Kwan Xu, Chengzhong Robotics Artificial Intelligence Navigating autonomous vehicles in open scenarios is a challenge due to the difficulties in handling unseen objects. Existing solutions either rely on small models that struggle with generalization or large models that are resource-intensive. While collaboration between the two offers a promising solution, the key challenge is deciding when and how to engage the large model. To address this issue, this paper proposes opportunistic collaborative planning (OCP), which seamlessly integrates efficient local models with powerful cloud models through two key innovations. First, we propose large vision model guided model predictive control (LVM-MPC), which leverages the cloud for LVM perception and decision making. The cloud output serves as a global guidance for a local MPC, thereby forming a closed-loop perception-to-control system. Second, to determine the best timing for large model query and service, we propose collaboration timing optimization (CTO), including object detection confidence thresholding (ODCT) and cloud forward simulation (CFS), to decide when to seek cloud assistance and when to offer cloud service. Extensive experiments show that the proposed OCP outperforms existing methods in terms of both navigation time and success rate. |
| title | Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2504.18057 |