Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913402139443200 |
|---|---|
| author | Barbato, Francesco Camuffo, Elena Milani, Simone Zanuttigh, Pietro |
| author_facet | Barbato, Francesco Camuffo, Elena Milani, Simone Zanuttigh, Pietro |
| contents | State-of-the-art multimodal semantic segmentation strategies combining LiDAR and color data are usually designed on top of asymmetric information-sharing schemes and assume that both modalities are always available. This strong assumption may not hold in real-world scenarios, where sensors are prone to failure or can face adverse conditions that make the acquired information unreliable. This problem is exacerbated when continual learning scenarios are considered since they have stringent data reliability constraints. In this work, we re-frame the task of multimodal semantic segmentation by enforcing a tightly coupled feature representation and a symmetric information-sharing scheme, which allows our approach to work even when one of the input modalities is missing. We also introduce an ad-hoc class-incremental continual learning scheme, proving our approach's effectiveness and reliability even in safety-critical settings, such as autonomous driving. We evaluate our approach on the SemanticKITTI dataset, achieving impressive performances. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_04702 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network Barbato, Francesco Camuffo, Elena Milani, Simone Zanuttigh, Pietro Computer Vision and Pattern Recognition State-of-the-art multimodal semantic segmentation strategies combining LiDAR and color data are usually designed on top of asymmetric information-sharing schemes and assume that both modalities are always available. This strong assumption may not hold in real-world scenarios, where sensors are prone to failure or can face adverse conditions that make the acquired information unreliable. This problem is exacerbated when continual learning scenarios are considered since they have stringent data reliability constraints. In this work, we re-frame the task of multimodal semantic segmentation by enforcing a tightly coupled feature representation and a symmetric information-sharing scheme, which allows our approach to work even when one of the input modalities is missing. We also introduce an ad-hoc class-incremental continual learning scheme, proving our approach's effectiveness and reliability even in safety-critical settings, such as autonomous driving. We evaluate our approach on the SemanticKITTI dataset, achieving impressive performances. |
| title | Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2308.04702 |