LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910766517452800 |
|---|---|
| author | Ayar, Fardin Javanmardi, Ehsan Tsukada, Manabu Javanmardi, Mahdi Rahmati, Mohammad |
| author_facet | Ayar, Fardin Javanmardi, Ehsan Tsukada, Manabu Javanmardi, Mahdi Rahmati, Mohammad |
| contents | Panoptic segmentation, which combines instance and semantic segmentation, has gained a lot of attention in autonomous vehicles, due to its comprehensive representation of the scene. This task can be applied for cameras and LiDAR sensors, but there has been a limited focus on combining both sensors to enhance image panoptic segmentation (PS). Although previous research has acknowledged the benefit of 3D data on camera-based scene perception, no specific study has explored the influence of 3D data on image and video panoptic segmentation (VPS).This work seeks to introduce a feature fusion module that enhances PS and VPS by fusing LiDAR and image data for autonomous vehicles. We also illustrate that, in addition to this fusion, our proposed model, which utilizes two simple modifications, can further deliver even more high-quality VPS without being trained on video data. The results demonstrate a substantial improvement in both the image and video panoptic segmentation evaluation metrics by up to 5 points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20881 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training Ayar, Fardin Javanmardi, Ehsan Tsukada, Manabu Javanmardi, Mahdi Rahmati, Mohammad Computer Vision and Pattern Recognition Panoptic segmentation, which combines instance and semantic segmentation, has gained a lot of attention in autonomous vehicles, due to its comprehensive representation of the scene. This task can be applied for cameras and LiDAR sensors, but there has been a limited focus on combining both sensors to enhance image panoptic segmentation (PS). Although previous research has acknowledged the benefit of 3D data on camera-based scene perception, no specific study has explored the influence of 3D data on image and video panoptic segmentation (VPS).This work seeks to introduce a feature fusion module that enhances PS and VPS by fusing LiDAR and image data for autonomous vehicles. We also illustrate that, in addition to this fusion, our proposed model, which utilizes two simple modifications, can further deliver even more high-quality VPS without being trained on video data. The results demonstrate a substantial improvement in both the image and video panoptic segmentation evaluation metrics by up to 5 points. |
| title | LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.20881 |