Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914946701328384 |
|---|---|
| author | Ding, Kairui Chen, Boyuan Su, Yuchen Gao, Huan-ang Jin, Bu Sima, Chonghao Zhang, Wuqiang Li, Xiaohui Barsch, Paul Li, Hongyang Zhao, Hao |
| author_facet | Ding, Kairui Chen, Boyuan Su, Yuchen Gao, Huan-ang Jin, Bu Sima, Chonghao Zhang, Wuqiang Li, Xiaohui Barsch, Paul Li, Hongyang Zhao, Hao |
| contents | End-to-end architectures in autonomous driving (AD) face a significant challenge in interpretability, impeding human-AI trust. Human-friendly natural language has been explored for tasks such as driving explanation and 3D captioning. However, previous works primarily focused on the paradigm of declarative interpretability, where the natural language interpretations are not grounded in the intermediate outputs of AD systems, making the interpretations only declarative. In contrast, aligned interpretability establishes a connection between language and the intermediate outputs of AD systems. Here we introduce Hint-AD, an integrated AD-language system that generates language aligned with the holistic perception-prediction-planning outputs of the AD model. By incorporating the intermediate outputs and a holistic token mixer sub-network for effective feature adaptation, Hint-AD achieves desirable accuracy, achieving state-of-the-art results in driving language tasks including driving explanation, 3D dense captioning, and command prediction. To facilitate further study on driving explanation task on nuScenes, we also introduce a human-labeled dataset, Nu-X. Codes, dataset, and models will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_06702 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving Ding, Kairui Chen, Boyuan Su, Yuchen Gao, Huan-ang Jin, Bu Sima, Chonghao Zhang, Wuqiang Li, Xiaohui Barsch, Paul Li, Hongyang Zhao, Hao Computer Vision and Pattern Recognition Artificial Intelligence End-to-end architectures in autonomous driving (AD) face a significant challenge in interpretability, impeding human-AI trust. Human-friendly natural language has been explored for tasks such as driving explanation and 3D captioning. However, previous works primarily focused on the paradigm of declarative interpretability, where the natural language interpretations are not grounded in the intermediate outputs of AD systems, making the interpretations only declarative. In contrast, aligned interpretability establishes a connection between language and the intermediate outputs of AD systems. Here we introduce Hint-AD, an integrated AD-language system that generates language aligned with the holistic perception-prediction-planning outputs of the AD model. By incorporating the intermediate outputs and a holistic token mixer sub-network for effective feature adaptation, Hint-AD achieves desirable accuracy, achieving state-of-the-art results in driving language tasks including driving explanation, 3D dense captioning, and command prediction. To facilitate further study on driving explanation task on nuScenes, we also introduce a human-labeled dataset, Nu-X. Codes, dataset, and models will be publicly available. |
| title | Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2409.06702 |