Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.14905 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917902542700544 |
|---|---|
| author | Anderson, Madeline Cha, Miriam Freeman, William T. Perron, J. Taylor Maidel, Nathaniel Cahoy, Kerri |
| author_facet | Anderson, Madeline Cha, Miriam Freeman, William T. Perron, J. Taylor Maidel, Nathaniel Cahoy, Kerri |
| contents | Vision language models have achieved impressive results across various fields. However, adoption in remote sensing remains limited, largely due to the scarcity of paired image-text data. To bridge this gap, synthetic caption generation has gained interest, traditionally relying on rule-based methods that use metadata or bounding boxes. While these approaches provide some description, they often lack the depth needed to capture complex wide-area scenes. Large language models (LLMs) offer a promising alternative for generating more descriptive captions, yet they can produce generic outputs and are prone to hallucination. In this paper, we propose a new method to enhance vision-language datasets for remote sensing by integrating maps as external data sources, enabling the generation of detailed, context-rich captions. Additionally, we present methods to measure and mitigate hallucinations in LLM-generated text. We introduce fMoW-mm, a multimodal dataset incorporating satellite imagery, maps, metadata, and text annotations. We demonstrate its effectiveness for automatic target recognition in few-shot settings, achieving superior performance compared to other vision-language remote sensing datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14905 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing Anderson, Madeline Cha, Miriam Freeman, William T. Perron, J. Taylor Maidel, Nathaniel Cahoy, Kerri Computer Vision and Pattern Recognition Vision language models have achieved impressive results across various fields. However, adoption in remote sensing remains limited, largely due to the scarcity of paired image-text data. To bridge this gap, synthetic caption generation has gained interest, traditionally relying on rule-based methods that use metadata or bounding boxes. While these approaches provide some description, they often lack the depth needed to capture complex wide-area scenes. Large language models (LLMs) offer a promising alternative for generating more descriptive captions, yet they can produce generic outputs and are prone to hallucination. In this paper, we propose a new method to enhance vision-language datasets for remote sensing by integrating maps as external data sources, enabling the generation of detailed, context-rich captions. Additionally, we present methods to measure and mitigate hallucinations in LLM-generated text. We introduce fMoW-mm, a multimodal dataset incorporating satellite imagery, maps, metadata, and text annotations. We demonstrate its effectiveness for automatic target recognition in few-shot settings, achieving superior performance compared to other vision-language remote sensing datasets. |
| title | Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.14905 |