GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915752554004480 |
|---|---|
| author | Sun, Qigan Zhang, Chaoning Zhang, Jianwei Wang, Xudong Xie, Jiehui Zheng, Pengcheng Wang, Haoyu Lee, Sungyoung Tai, Chi-lok Andy Yang, Yang Shen, Heng Tao |
| author_facet | Sun, Qigan Zhang, Chaoning Zhang, Jianwei Wang, Xudong Xie, Jiehui Zheng, Pengcheng Wang, Haoyu Lee, Sungyoung Tai, Chi-lok Andy Yang, Yang Shen, Heng Tao |
| contents | In recent years, Multimodal Large Language Models (MLLMs) have made significant progress in visual question answering tasks. However, directly applying existing fine-tuning methods to remote sensing (RS) images often leads to issues such as overfitting on background noise or neglecting target details. This is primarily due to the large-scale variations, sparse target distributions, and complex regional semantic features inherent in RS images. These challenges limit the effectiveness of MLLMs in RS tasks. To address these challenges, we propose a parameter-efficient fine-tuning (PEFT) strategy called Guided Region-Aware Sparse Prompting (GRASP). GRASP introduces spatially structured soft prompts associated with spatial blocks extracted from a frozen visual token grid. Through a question-guided sparse fusion mechanism, GRASP dynamically aggregates task-specific context into a compact global prompt, enabling the model to focus on relevant regions while filtering out background noise. Extensive experiments on multiple RSVQA benchmarks show that GRASP achieves competitive performance compared to existing fine-tuning and prompt-based methods while maintaining high parameter efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_17089 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing Sun, Qigan Zhang, Chaoning Zhang, Jianwei Wang, Xudong Xie, Jiehui Zheng, Pengcheng Wang, Haoyu Lee, Sungyoung Tai, Chi-lok Andy Yang, Yang Shen, Heng Tao Computer Vision and Pattern Recognition In recent years, Multimodal Large Language Models (MLLMs) have made significant progress in visual question answering tasks. However, directly applying existing fine-tuning methods to remote sensing (RS) images often leads to issues such as overfitting on background noise or neglecting target details. This is primarily due to the large-scale variations, sparse target distributions, and complex regional semantic features inherent in RS images. These challenges limit the effectiveness of MLLMs in RS tasks. To address these challenges, we propose a parameter-efficient fine-tuning (PEFT) strategy called Guided Region-Aware Sparse Prompting (GRASP). GRASP introduces spatially structured soft prompts associated with spatial blocks extracted from a frozen visual token grid. Through a question-guided sparse fusion mechanism, GRASP dynamically aggregates task-specific context into a compact global prompt, enabling the model to focus on relevant regions while filtering out background noise. Extensive experiments on multiple RSVQA benchmarks show that GRASP achieves competitive performance compared to existing fine-tuning and prompt-based methods while maintaining high parameter efficiency. |
| title | GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.17089 |