Contrastive Pretraining for Visual Concept Explanations of Socioeconomic Outcomes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916284712615936 |
|---|---|
| author | Obadic, Ivica Levering, Alex Pennig, Lars Oliveira, Dario Marcos, Diego Zhu, Xiaoxiang |
| author_facet | Obadic, Ivica Levering, Alex Pennig, Lars Oliveira, Dario Marcos, Diego Zhu, Xiaoxiang |
| contents | Predicting socioeconomic indicators from satellite imagery with deep learning has become an increasingly popular research direction. Post-hoc concept-based explanations can be an important step towards broader adoption of these models in policy-making as they enable the interpretation of socioeconomic outcomes based on visual concepts that are intuitive to humans. In this paper, we study the interplay between representation learning using an additional task-specific contrastive loss and post-hoc concept explainability for socioeconomic studies. Our results on two different geographical locations and tasks indicate that the task-specific pretraining imposes a continuous ordering of the latent space embeddings according to the socioeconomic outcomes. This improves the model's interpretability as it enables the latent space of the model to associate concepts encoding typical urban and natural area patterns with continuous intervals of socioeconomic outcomes. Further, we illustrate how analyzing the model's conceptual sensitivity for the intervals of socioeconomic outcomes can shed light on new insights for urban studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_09768 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Contrastive Pretraining for Visual Concept Explanations of Socioeconomic Outcomes Obadic, Ivica Levering, Alex Pennig, Lars Oliveira, Dario Marcos, Diego Zhu, Xiaoxiang Computer Vision and Pattern Recognition Predicting socioeconomic indicators from satellite imagery with deep learning has become an increasingly popular research direction. Post-hoc concept-based explanations can be an important step towards broader adoption of these models in policy-making as they enable the interpretation of socioeconomic outcomes based on visual concepts that are intuitive to humans. In this paper, we study the interplay between representation learning using an additional task-specific contrastive loss and post-hoc concept explainability for socioeconomic studies. Our results on two different geographical locations and tasks indicate that the task-specific pretraining imposes a continuous ordering of the latent space embeddings according to the socioeconomic outcomes. This improves the model's interpretability as it enables the latent space of the model to associate concepts encoding typical urban and natural area patterns with continuous intervals of socioeconomic outcomes. Further, we illustrate how analyzing the model's conceptual sensitivity for the intervals of socioeconomic outcomes can shed light on new insights for urban studies. |
| title | Contrastive Pretraining for Visual Concept Explanations of Socioeconomic Outcomes |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.09768 |