Enhancing End-to-End Autonomous Driving with Latent World Model
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913711803858944 |
|---|---|
| author | Li, Yingyan Fan, Lue He, Jiawei Wang, Yuqi Chen, Yuntao Zhang, Zhaoxiang Tan, Tieniu |
| author_facet | Li, Yingyan Fan, Lue He, Jiawei Wang, Yuqi Chen, Yuntao Zhang, Zhaoxiang Tan, Tieniu |
| contents | In autonomous driving, end-to-end planners directly utilize raw sensor data, enabling them to extract richer scene features and reduce information loss compared to traditional planners. This raises a crucial research question: how can we develop better scene feature representations to fully leverage sensor data in end-to-end driving? Self-supervised learning methods show great success in learning rich feature representations in NLP and computer vision. Inspired by this, we propose a novel self-supervised learning approach using the LAtent World model (LAW) for end-to-end driving. LAW predicts future scene features based on current features and ego trajectories. This self-supervised task can be seamlessly integrated into perception-free and perception-based frameworks, improving scene feature learning and optimizing trajectory prediction. LAW achieves state-of-the-art performance across multiple benchmarks, including real-world open-loop benchmark nuScenes, NAVSIM, and simulator-based closed-loop benchmark CARLA. The code is released at https://github.com/BraveGroup/LAW. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08481 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing End-to-End Autonomous Driving with Latent World Model Li, Yingyan Fan, Lue He, Jiawei Wang, Yuqi Chen, Yuntao Zhang, Zhaoxiang Tan, Tieniu Computer Vision and Pattern Recognition In autonomous driving, end-to-end planners directly utilize raw sensor data, enabling them to extract richer scene features and reduce information loss compared to traditional planners. This raises a crucial research question: how can we develop better scene feature representations to fully leverage sensor data in end-to-end driving? Self-supervised learning methods show great success in learning rich feature representations in NLP and computer vision. Inspired by this, we propose a novel self-supervised learning approach using the LAtent World model (LAW) for end-to-end driving. LAW predicts future scene features based on current features and ego trajectories. This self-supervised task can be seamlessly integrated into perception-free and perception-based frameworks, improving scene feature learning and optimizing trajectory prediction. LAW achieves state-of-the-art performance across multiple benchmarks, including real-world open-loop benchmark nuScenes, NAVSIM, and simulator-based closed-loop benchmark CARLA. The code is released at https://github.com/BraveGroup/LAW. |
| title | Enhancing End-to-End Autonomous Driving with Latent World Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.08481 |