WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weather
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909194681057280 |
|---|---|
| author | Gella, Blake Zhang, Howard Upadhyay, Rishi Chang, Tiffany Wei, Nathan Waliman, Matthew Ba, Yunhao de Melo, Celso Wong, Alex Kadambi, Achuta |
| author_facet | Gella, Blake Zhang, Howard Upadhyay, Rishi Chang, Tiffany Wei, Nathan Waliman, Matthew Ba, Yunhao de Melo, Celso Wong, Alex Kadambi, Achuta |
| contents | We propose a method to infer semantic segmentation maps from images captured under adverse weather conditions. We begin by examining existing models on images degraded by weather conditions such as rain, fog, or snow, and found that they exhibit a large performance drop as compared to those captured under clear weather. To control for changes in scene structures, we propose WeatherProof, the first semantic segmentation dataset with accurate clear and adverse weather image pairs that share an underlying scene. Through this dataset, we analyze the error modes in existing models and found that they were sensitive to the highly complex combination of different weather effects induced on the image during capture. To improve robustness, we propose a way to use language as guidance by identifying contributions of adverse weather conditions and injecting that as "side information". Models trained using our language guidance exhibit performance gains by up to 10.2% in mIoU on WeatherProof, up to 8.44% in mIoU on the widely used ACDC dataset compared to standard training techniques, and up to 6.21% in mIoU on the ACDC dataset as compared to previous SOTA methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14874 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weather Gella, Blake Zhang, Howard Upadhyay, Rishi Chang, Tiffany Wei, Nathan Waliman, Matthew Ba, Yunhao de Melo, Celso Wong, Alex Kadambi, Achuta Computer Vision and Pattern Recognition Machine Learning We propose a method to infer semantic segmentation maps from images captured under adverse weather conditions. We begin by examining existing models on images degraded by weather conditions such as rain, fog, or snow, and found that they exhibit a large performance drop as compared to those captured under clear weather. To control for changes in scene structures, we propose WeatherProof, the first semantic segmentation dataset with accurate clear and adverse weather image pairs that share an underlying scene. Through this dataset, we analyze the error modes in existing models and found that they were sensitive to the highly complex combination of different weather effects induced on the image during capture. To improve robustness, we propose a way to use language as guidance by identifying contributions of adverse weather conditions and injecting that as "side information". Models trained using our language guidance exhibit performance gains by up to 10.2% in mIoU on WeatherProof, up to 8.44% in mIoU on the widely used ACDC dataset compared to standard training techniques, and up to 6.21% in mIoU on the ACDC dataset as compared to previous SOTA methods. |
| title | WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weather |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2403.14874 |