A New Dataset and Framework for Robust Road Surface Classification via Camera-IMU Fusion
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917232096837632 |
|---|---|
| author | Costa, Willams de Lima de Souza, Thifany Ketuli Silva Silva, Jonas Ferreira Pereira, Carlos Gabriel Bezerra Nova, Bruno Reis Vila Brito, Leonardo Silvino Leoni, Rafael Raider Filho, Juliano Silva Ferreira, Valter Neto, Sibele Miguel Soares Uehara, Samantha Amaral, Daniel Giacometti Teixeira, João Marcelo Teichrieb, Veronica de Araújo, Cristiano Coelho |
| author_facet | Costa, Willams de Lima de Souza, Thifany Ketuli Silva Silva, Jonas Ferreira Pereira, Carlos Gabriel Bezerra Nova, Bruno Reis Vila Brito, Leonardo Silvino Leoni, Rafael Raider Filho, Juliano Silva Ferreira, Valter Neto, Sibele Miguel Soares Uehara, Samantha Amaral, Daniel Giacometti Teixeira, João Marcelo Teichrieb, Veronica de Araújo, Cristiano Coelho |
| contents | Road surface classification (RSC) is a key enabler for environment-aware predictive maintenance systems. However, existing RSC techniques often fail to generalize beyond narrow operational conditions due to limited sensing modalities and datasets that lack environmental diversity. This work addresses these limitations by introducing a multimodal framework that fuses images and inertial measurements using a lightweight bidirectional cross-attention module followed by an adaptive gating layer that adjusts modality contributions under domain shifts. Given the limitations of current benchmarks, especially regarding lack of variability, we introduce ROAD, a new dataset composed of three complementary subsets: (i) real-world multimodal recordings with RGB-IMU streams synchronized using a gold-standard industry datalogger, captured across diverse lighting, weather, and surface conditions; (ii) a large vision-only subset designed to assess robustness under adverse illumination and heterogeneous capture setups; and (iii) a synthetic subset generated to study out-of-distribution generalization in scenarios difficult to obtain in practice. Experiments show that our method achieves a +1.4 pp improvement over the previous state-of-the-art on the PVS benchmark and an +11.6 pp improvement on our multimodal ROAD subset, with consistently higher F1-scores on minority classes. The framework also demonstrates stable performance across challenging visual conditions, including nighttime, heavy rain, and mixed-surface transitions. These findings indicate that combining affordable camera and IMU sensors with multimodal attention mechanisms provides a scalable, robust foundation for road surface understanding, particularly relevant for regions where environmental variability and cost constraints limit the adoption of high-end sensing suites. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20847 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A New Dataset and Framework for Robust Road Surface Classification via Camera-IMU Fusion Costa, Willams de Lima de Souza, Thifany Ketuli Silva Silva, Jonas Ferreira Pereira, Carlos Gabriel Bezerra Nova, Bruno Reis Vila Brito, Leonardo Silvino Leoni, Rafael Raider Filho, Juliano Silva Ferreira, Valter Neto, Sibele Miguel Soares Uehara, Samantha Amaral, Daniel Giacometti Teixeira, João Marcelo Teichrieb, Veronica de Araújo, Cristiano Coelho Computer Vision and Pattern Recognition Artificial Intelligence Road surface classification (RSC) is a key enabler for environment-aware predictive maintenance systems. However, existing RSC techniques often fail to generalize beyond narrow operational conditions due to limited sensing modalities and datasets that lack environmental diversity. This work addresses these limitations by introducing a multimodal framework that fuses images and inertial measurements using a lightweight bidirectional cross-attention module followed by an adaptive gating layer that adjusts modality contributions under domain shifts. Given the limitations of current benchmarks, especially regarding lack of variability, we introduce ROAD, a new dataset composed of three complementary subsets: (i) real-world multimodal recordings with RGB-IMU streams synchronized using a gold-standard industry datalogger, captured across diverse lighting, weather, and surface conditions; (ii) a large vision-only subset designed to assess robustness under adverse illumination and heterogeneous capture setups; and (iii) a synthetic subset generated to study out-of-distribution generalization in scenarios difficult to obtain in practice. Experiments show that our method achieves a +1.4 pp improvement over the previous state-of-the-art on the PVS benchmark and an +11.6 pp improvement on our multimodal ROAD subset, with consistently higher F1-scores on minority classes. The framework also demonstrates stable performance across challenging visual conditions, including nighttime, heavy rain, and mixed-surface transitions. These findings indicate that combining affordable camera and IMU sensors with multimodal attention mechanisms provides a scalable, robust foundation for road surface understanding, particularly relevant for regions where environmental variability and cost constraints limit the adoption of high-end sensing suites. |
| title | A New Dataset and Framework for Robust Road Surface Classification via Camera-IMU Fusion |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2601.20847 |