Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.08708 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911499337859072 |
|---|---|
| author | Li, Haoyang Wang, Liang Zhou, Siyu Sun, Jiacheng Jiang, Jing Wang, Chao Long, Guodong Peng, Yan |
| author_facet | Li, Haoyang Wang, Liang Zhou, Siyu Sun, Jiacheng Jiang, Jing Wang, Chao Long, Guodong Peng, Yan |
| contents | CLIP-based prompt tuning enables pretrained Vision-Language Models (VLMs) to efficiently adapt to downstream tasks. Although existing studies have made significant progress, they pay limited attention to changes in the internal attention representations of VLMs during the tuning process. In this paper, we attribute the failure modes of prompt tuning predictions to shifts in foreground attention of the visual encoder, and propose Foreground View-Guided Prompt Tuning (FVG-PT), an adaptive plug-and-play foreground attention guidance module, to alleviate the shifts. Concretely, FVG-PT introduces a learnable Foreground Reliability Gate to automatically enhance the foreground view quality, applies a Foreground Distillation Compensation module to guide visual attention toward the foreground, and further introduces a Prior Calibration module to mitigate generalization degradation caused by excessive focus on the foreground. Experiments on multiple backbone models and datasets show the effectiveness and compatibility of FVG-PT. Codes are available at: https://github.com/JREion/FVG-PT |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08708 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Models Li, Haoyang Wang, Liang Zhou, Siyu Sun, Jiacheng Jiang, Jing Wang, Chao Long, Guodong Peng, Yan Computer Vision and Pattern Recognition CLIP-based prompt tuning enables pretrained Vision-Language Models (VLMs) to efficiently adapt to downstream tasks. Although existing studies have made significant progress, they pay limited attention to changes in the internal attention representations of VLMs during the tuning process. In this paper, we attribute the failure modes of prompt tuning predictions to shifts in foreground attention of the visual encoder, and propose Foreground View-Guided Prompt Tuning (FVG-PT), an adaptive plug-and-play foreground attention guidance module, to alleviate the shifts. Concretely, FVG-PT introduces a learnable Foreground Reliability Gate to automatically enhance the foreground view quality, applies a Foreground Distillation Compensation module to guide visual attention toward the foreground, and further introduces a Prior Calibration module to mitigate generalization degradation caused by excessive focus on the foreground. Experiments on multiple backbone models and datasets show the effectiveness and compatibility of FVG-PT. Codes are available at: https://github.com/JREion/FVG-PT |
| title | FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.08708 |