Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.12325 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917490371592192 |
|---|---|
| author | Zhu, Hao Jin, Shuo Liao, Wenbin Xiao, Jiayu Zhu, Yan Yu, Siyue Dai, Feng |
| author_facet | Zhu, Hao Jin, Shuo Liao, Wenbin Xiao, Jiayu Zhu, Yan Yu, Siyue Dai, Feng |
| contents | Pursuing training-free open-vocabulary semantic segmentation in an efficient and generalizable manner remains challenging due to the deep-seated spatial bias in CLIP. To overcome the limitations of existing solutions, this work moves beyond the CLIP-based paradigm and harnesses the recent spatially-aware dino$.$txt framework to facilitate more efficient and high-quality dense prediction. While dino$.$txt exhibits robust spatial awareness, we find that the semantic ambiguity of text queries gives rise to severe mismatch within its dense cross-modal interactions. To address this, we introduce Visual-guided Prompt evolution (VIP) to rectify the semantic expressiveness of text queries in dino$.$txt, unleashing its potential for fine-grained object perception. Towards this end, VIP integrates alias expansion with a visual-guided distillation mechanism to mine valuable semantic cues, which are robustly aggregated in a saliency-aware manner to yield a high-fidelity prediction. Extensive evaluations demonstrate that VIP: 1. surpasses the top-leading methods by 1.4%-8.4% average mIoU, 2. generalizes well to diverse challenging domains, and 3. requires marginal inference time and memory overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_12325 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inference Zhu, Hao Jin, Shuo Liao, Wenbin Xiao, Jiayu Zhu, Yan Yu, Siyue Dai, Feng Computer Vision and Pattern Recognition Pursuing training-free open-vocabulary semantic segmentation in an efficient and generalizable manner remains challenging due to the deep-seated spatial bias in CLIP. To overcome the limitations of existing solutions, this work moves beyond the CLIP-based paradigm and harnesses the recent spatially-aware dino$.$txt framework to facilitate more efficient and high-quality dense prediction. While dino$.$txt exhibits robust spatial awareness, we find that the semantic ambiguity of text queries gives rise to severe mismatch within its dense cross-modal interactions. To address this, we introduce Visual-guided Prompt evolution (VIP) to rectify the semantic expressiveness of text queries in dino$.$txt, unleashing its potential for fine-grained object perception. Towards this end, VIP integrates alias expansion with a visual-guided distillation mechanism to mine valuable semantic cues, which are robustly aggregated in a saliency-aware manner to yield a high-fidelity prediction. Extensive evaluations demonstrate that VIP: 1. surpasses the top-leading methods by 1.4%-8.4% average mIoU, 2. generalizes well to diverse challenging domains, and 3. requires marginal inference time and memory overhead. |
| title | VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inference |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.12325 |