A Multimodal Hybrid Late-Cascade Fusion Network for Enhanced 3D Object Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917203557744640 |
|---|---|
| author | Sgaravatti, Carlo Basla, Roberto Pieroni, Riccardo Corno, Matteo Savaresi, Sergio M. Magri, Luca Boracchi, Giacomo |
| author_facet | Sgaravatti, Carlo Basla, Roberto Pieroni, Riccardo Corno, Matteo Savaresi, Sergio M. Magri, Luca Boracchi, Giacomo |
| contents | We present a new way to detect 3D objects from multimodal inputs, leveraging both LiDAR and RGB cameras in a hybrid late-cascade scheme, that combines an RGB detection network and a 3D LiDAR detector. We exploit late fusion principles to reduce LiDAR False Positives, matching LiDAR detections with RGB ones by projecting the LiDAR bounding boxes on the image. We rely on cascade fusion principles to recover LiDAR False Negatives leveraging epipolar constraints and frustums generated by RGB detections of separate views. Our solution can be plugged on top of any underlying single-modal detectors, enabling a flexible training process that can take advantage of pre-trained LiDAR and RGB detectors, or train the two branches separately. We evaluate our results on the KITTI object detection benchmark, showing significant performance improvements, especially for the detection of Pedestrians and Cyclists. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18419 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Multimodal Hybrid Late-Cascade Fusion Network for Enhanced 3D Object Detection Sgaravatti, Carlo Basla, Roberto Pieroni, Riccardo Corno, Matteo Savaresi, Sergio M. Magri, Luca Boracchi, Giacomo Computer Vision and Pattern Recognition Artificial Intelligence Robotics We present a new way to detect 3D objects from multimodal inputs, leveraging both LiDAR and RGB cameras in a hybrid late-cascade scheme, that combines an RGB detection network and a 3D LiDAR detector. We exploit late fusion principles to reduce LiDAR False Positives, matching LiDAR detections with RGB ones by projecting the LiDAR bounding boxes on the image. We rely on cascade fusion principles to recover LiDAR False Negatives leveraging epipolar constraints and frustums generated by RGB detections of separate views. Our solution can be plugged on top of any underlying single-modal detectors, enabling a flexible training process that can take advantage of pre-trained LiDAR and RGB detectors, or train the two branches separately. We evaluate our results on the KITTI object detection benchmark, showing significant performance improvements, especially for the detection of Pedestrians and Cyclists. |
| title | A Multimodal Hybrid Late-Cascade Fusion Network for Enhanced 3D Object Detection |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2504.18419 |