DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908911468019712 |
|---|---|
| author | Tao, Mingzhe Liu, Ruiping Zheng, Junwei Chen, Yufan Ying, Kedi Sarfraz, M. Saquib Yang, Kailun Zhang, Jiaming Stiefelhagen, Rainer |
| author_facet | Tao, Mingzhe Liu, Ruiping Zheng, Junwei Chen, Yufan Ying, Kedi Sarfraz, M. Saquib Yang, Kailun Zhang, Jiaming Stiefelhagen, Rainer |
| contents | Fusing sensors with complementary modalities is crucial for maintaining a stable and comprehensive understanding of abnormal driving scenes. However, Multimodal Large Language Models (MLLMs) are underexplored for leveraging multi-sensor information to understand adverse driving scenarios in autonomous vehicles. To address this gap, we propose the DriveXQA, a multimodal dataset for autonomous driving VQA. In addition to four visual modalities, five sensor failure cases, and five weather conditions, it includes $102,505$ QA pairs categorized into three types: global scene level, allocentric level, and ego-vehicle centric level. Since no existing MLLM framework adopts multiple complementary visual modalities as input, we design MVX-LLM, a token-efficient architecture with a Dual Cross-Attention (DCA) projector that fuses the modalities to alleviate information redundancy. Experiments demonstrate that our DCA achieves improved performance under challenging conditions such as foggy (GPTScore: $53.5$ vs. $25.1$ for the baseline). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11380 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding Tao, Mingzhe Liu, Ruiping Zheng, Junwei Chen, Yufan Ying, Kedi Sarfraz, M. Saquib Yang, Kailun Zhang, Jiaming Stiefelhagen, Rainer Computer Vision and Pattern Recognition Fusing sensors with complementary modalities is crucial for maintaining a stable and comprehensive understanding of abnormal driving scenes. However, Multimodal Large Language Models (MLLMs) are underexplored for leveraging multi-sensor information to understand adverse driving scenarios in autonomous vehicles. To address this gap, we propose the DriveXQA, a multimodal dataset for autonomous driving VQA. In addition to four visual modalities, five sensor failure cases, and five weather conditions, it includes $102,505$ QA pairs categorized into three types: global scene level, allocentric level, and ego-vehicle centric level. Since no existing MLLM framework adopts multiple complementary visual modalities as input, we design MVX-LLM, a token-efficient architecture with a Dual Cross-Attention (DCA) projector that fuses the modalities to alleviate information redundancy. Experiments demonstrate that our DCA achieves improved performance under challenging conditions such as foggy (GPTScore: $53.5$ vs. $25.1$ for the baseline). |
| title | DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.11380 |