GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912687112323072 |
|---|---|
| author | Wang, Fengxiang Chen, Mingshuo Li, Yueying Wang, Di Wang, Haotian Guo, Zonghao Wang, Zefan Shan, Boqi Lan, Long Wang, Yulin Wang, Hongzhen Yang, Wenjing Du, Bo Zhang, Jing |
| author_facet | Wang, Fengxiang Chen, Mingshuo Li, Yueying Wang, Di Wang, Haotian Guo, Zonghao Wang, Zefan Shan, Boqi Lan, Long Wang, Yulin Wang, Hongzhen Yang, Wenjing Du, Bo Zhang, Jing |
| contents | Ultra-high-resolution (UHR) remote sensing (RS) imagery offers valuable data for Earth observation but pose challenges for existing multimodal foundation models due to two key bottlenecks: (1) limited availability of UHR training data, and (2) token explosion caused by the large image size. To address data scarcity, we introduce SuperRS-VQA (avg. 8,376$\times$8,376) and HighRS-VQA (avg. 2,000$\times$1,912), the highest-resolution vision-language datasets in RS to date, covering 22 real-world dialogue tasks. To mitigate token explosion, our pilot studies reveal significant redundancy in RS images: crucial information is concentrated in a small subset of object-centric tokens, while pruning background tokens (e.g., ocean or forest) can even improve performance. Motivated by these findings, we propose two strategies: Background Token Pruning and Anchored Token Selection, to reduce the memory footprint while preserving key semantics.Integrating these techniques, we introduce GeoLLaVA-8K, the first RS-focused multimodal large language model capable of handling inputs up to 8K$\times$8K resolution, built on the LLaVA framework. Trained on SuperRS-VQA and HighRS-VQA, GeoLLaVA-8K sets a new state-of-the-art on the XLRS-Bench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21375 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution Wang, Fengxiang Chen, Mingshuo Li, Yueying Wang, Di Wang, Haotian Guo, Zonghao Wang, Zefan Shan, Boqi Lan, Long Wang, Yulin Wang, Hongzhen Yang, Wenjing Du, Bo Zhang, Jing Computer Vision and Pattern Recognition Ultra-high-resolution (UHR) remote sensing (RS) imagery offers valuable data for Earth observation but pose challenges for existing multimodal foundation models due to two key bottlenecks: (1) limited availability of UHR training data, and (2) token explosion caused by the large image size. To address data scarcity, we introduce SuperRS-VQA (avg. 8,376$\times$8,376) and HighRS-VQA (avg. 2,000$\times$1,912), the highest-resolution vision-language datasets in RS to date, covering 22 real-world dialogue tasks. To mitigate token explosion, our pilot studies reveal significant redundancy in RS images: crucial information is concentrated in a small subset of object-centric tokens, while pruning background tokens (e.g., ocean or forest) can even improve performance. Motivated by these findings, we propose two strategies: Background Token Pruning and Anchored Token Selection, to reduce the memory footprint while preserving key semantics.Integrating these techniques, we introduce GeoLLaVA-8K, the first RS-focused multimodal large language model capable of handling inputs up to 8K$\times$8K resolution, built on the LLaVA framework. Trained on SuperRS-VQA and HighRS-VQA, GeoLLaVA-8K sets a new state-of-the-art on the XLRS-Bench. |
| title | GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.21375 |