123D: Unifying Multi-Modal Autonomous Driving Data at Scale
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914545486790656 |
|---|---|
| author | Dauner, Daniel Charraut, Valentin Berle, Bastian Li, Tianyu Nguyen, Long Wang, Jiabao Jing, Changhui Igl, Maximilian Caesar, Holger Ivanovic, Boris Liao, Yiyi Geiger, Andreas Chitta, Kashyap |
| author_facet | Dauner, Daniel Charraut, Valentin Berle, Bastian Li, Tianyu Nguyen, Long Wang, Jiabao Jing, Changhui Igl, Maximilian Caesar, Holger Ivanovic, Boris Liao, Yiyi Geiger, Andreas Chitta, Kashyap |
| contents | The pursuit of autonomous driving has produced one of the richest sensor data collections in all of robotics. However, its scale and diversity remain largely untapped. Each dataset adopts different 2D and 3D modalities, such as cameras, lidar, ego states, annotations, traffic lights, and HD maps, with different rates and synchronization schemes. They come in fragmented formats requiring complex dependencies that cannot natively coexist in the same development environment. Further, major inconsistencies in annotation conventions prevent training or measuring generalization across multiple datasets. We present 123D, an open-source framework that unifies such multi-modal driving data through a single API. To handle synchronization, we store each modality as an independent timestamped event stream with no prescribed rate, enabling synchronous or asynchronous access across arbitrary datasets. Using 123D, we consolidate eight real-world driving datasets spanning 3,300 hours and 90,000 kilometers, together with a synthetic dataset with configurable collection scripts, and provide tools for data analysis and visualization. We conduct a systematic study comparing annotation statistics and assessing each dataset's pose and calibration accuracy. Further, we showcase two applications 123D enables: cross-dataset 3D object detection transfer and reinforcement learning for planning, and offer recommendations for future directions. Code and documentation are available at https://github.com/kesai-labs/py123d. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_08084 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | 123D: Unifying Multi-Modal Autonomous Driving Data at Scale Dauner, Daniel Charraut, Valentin Berle, Bastian Li, Tianyu Nguyen, Long Wang, Jiabao Jing, Changhui Igl, Maximilian Caesar, Holger Ivanovic, Boris Liao, Yiyi Geiger, Andreas Chitta, Kashyap Robotics Computer Vision and Pattern Recognition The pursuit of autonomous driving has produced one of the richest sensor data collections in all of robotics. However, its scale and diversity remain largely untapped. Each dataset adopts different 2D and 3D modalities, such as cameras, lidar, ego states, annotations, traffic lights, and HD maps, with different rates and synchronization schemes. They come in fragmented formats requiring complex dependencies that cannot natively coexist in the same development environment. Further, major inconsistencies in annotation conventions prevent training or measuring generalization across multiple datasets. We present 123D, an open-source framework that unifies such multi-modal driving data through a single API. To handle synchronization, we store each modality as an independent timestamped event stream with no prescribed rate, enabling synchronous or asynchronous access across arbitrary datasets. Using 123D, we consolidate eight real-world driving datasets spanning 3,300 hours and 90,000 kilometers, together with a synthetic dataset with configurable collection scripts, and provide tools for data analysis and visualization. We conduct a systematic study comparing annotation statistics and assessing each dataset's pose and calibration accuracy. Further, we showcase two applications 123D enables: cross-dataset 3D object detection transfer and reinforcement learning for planning, and offer recommendations for future directions. Code and documentation are available at https://github.com/kesai-labs/py123d. |
| title | 123D: Unifying Multi-Modal Autonomous Driving Data at Scale |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.08084 |