WildSAT: Learning Satellite Image Representations from Wildlife Observations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913979482243072 |
|---|---|
| author | Daroya, Rangel Cole, Elijah Mac Aodha, Oisin Van Horn, Grant Maji, Subhransu |
| author_facet | Daroya, Rangel Cole, Elijah Mac Aodha, Oisin Van Horn, Grant Maji, Subhransu |
| contents | Species distributions encode valuable ecological and environmental information, yet their potential for guiding representation learning in remote sensing remains underexplored. We introduce WildSAT, which pairs satellite images with millions of geo-tagged wildlife observations readily-available on citizen science platforms. WildSAT employs a contrastive learning approach that jointly leverages satellite images, species occurrence maps, and textual habitat descriptions to train or fine-tune models. This approach significantly improves performance on diverse satellite image recognition tasks, outperforming both ImageNet-pretrained models and satellite-specific baselines. Additionally, by aligning visual and textual information, WildSAT enables zero-shot retrieval, allowing users to search geographic locations based on textual descriptions. WildSAT surpasses recent cross-modal learning methods, including approaches that align satellite images with ground imagery or wildlife photos, demonstrating the advantages of our approach. Finally, we analyze the impact of key design choices and highlight the broad applicability of WildSAT to remote sensing and biodiversity monitoring. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_14428 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | WildSAT: Learning Satellite Image Representations from Wildlife Observations Daroya, Rangel Cole, Elijah Mac Aodha, Oisin Van Horn, Grant Maji, Subhransu Computer Vision and Pattern Recognition Machine Learning Quantitative Methods Species distributions encode valuable ecological and environmental information, yet their potential for guiding representation learning in remote sensing remains underexplored. We introduce WildSAT, which pairs satellite images with millions of geo-tagged wildlife observations readily-available on citizen science platforms. WildSAT employs a contrastive learning approach that jointly leverages satellite images, species occurrence maps, and textual habitat descriptions to train or fine-tune models. This approach significantly improves performance on diverse satellite image recognition tasks, outperforming both ImageNet-pretrained models and satellite-specific baselines. Additionally, by aligning visual and textual information, WildSAT enables zero-shot retrieval, allowing users to search geographic locations based on textual descriptions. WildSAT surpasses recent cross-modal learning methods, including approaches that align satellite images with ground imagery or wildlife photos, demonstrating the advantages of our approach. Finally, we analyze the impact of key design choices and highlight the broad applicability of WildSAT to remote sensing and biodiversity monitoring. |
| title | WildSAT: Learning Satellite Image Representations from Wildlife Observations |
| topic | Computer Vision and Pattern Recognition Machine Learning Quantitative Methods |
| url | https://arxiv.org/abs/2412.14428 |