LVIC: Multi-modality segmentation by Lifting Visual Info as Cue
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914708027604992 |
|---|---|
| author | Dong, Zichao Pang, Bowen Huang, Xufeng Ji, Hang Zhan, Xin Chen, Junbo |
| author_facet | Dong, Zichao Pang, Bowen Huang, Xufeng Ji, Hang Zhan, Xin Chen, Junbo |
| contents | Multi-modality fusion is proven an effective method for 3d perception for autonomous driving. However, most current multi-modality fusion pipelines for LiDAR semantic segmentation have complicated fusion mechanisms. Point painting is a quite straight forward method which directly bind LiDAR points with visual information. Unfortunately, previous point painting like methods suffer from projection error between camera and LiDAR. In our experiments, we find that this projection error is the devil in point painting. As a result of that, we propose a depth aware point painting mechanism, which significantly boosts the multi-modality fusion. Apart from that, we take a deeper look at the desired visual feature for LiDAR to operate semantic segmentation. By Lifting Visual Information as Cue, LVIC ranks 1st on nuScenes LiDAR semantic segmentation benchmark. Our experiments show the robustness and effectiveness. Codes would be make publicly available soon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_05159 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LVIC: Multi-modality segmentation by Lifting Visual Info as Cue Dong, Zichao Pang, Bowen Huang, Xufeng Ji, Hang Zhan, Xin Chen, Junbo Computer Vision and Pattern Recognition Multi-modality fusion is proven an effective method for 3d perception for autonomous driving. However, most current multi-modality fusion pipelines for LiDAR semantic segmentation have complicated fusion mechanisms. Point painting is a quite straight forward method which directly bind LiDAR points with visual information. Unfortunately, previous point painting like methods suffer from projection error between camera and LiDAR. In our experiments, we find that this projection error is the devil in point painting. As a result of that, we propose a depth aware point painting mechanism, which significantly boosts the multi-modality fusion. Apart from that, we take a deeper look at the desired visual feature for LiDAR to operate semantic segmentation. By Lifting Visual Information as Cue, LVIC ranks 1st on nuScenes LiDAR semantic segmentation benchmark. Our experiments show the robustness and effectiveness. Codes would be make publicly available soon. |
| title | LVIC: Multi-modality segmentation by Lifting Visual Info as Cue |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.05159 |