Not Just What's There: Enabling CLIP to Comprehend Negated Visual Descriptions Without Fine-tuning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910031620866048 |
|---|---|
| author | Xiao, Junhao Wu, Zhiyu Lin, Hao Chen, Yi Liu, Yahui Zhao, Xiaoran Wang, Zixu He, Zejiang |
| author_facet | Xiao, Junhao Wu, Zhiyu Lin, Hao Chen, Yi Liu, Yahui Zhao, Xiaoran Wang, Zixu He, Zejiang |
| contents | Vision-Language Models (VLMs) like CLIP struggle to understand negation, often embedding affirmatives and negatives similarly (e.g., matching "no dog" with dog images). Existing methods refine negation understanding via fine-tuning CLIP's text encoder, risking overfitting. In this work, we propose CLIPGlasses, a plug-and-play framework that enhances CLIP's ability to comprehend negated visual descriptions. CLIPGlasses adopts a dual-stage design: a Lens module disentangles negated semantics from text embeddings, and a Frame module predicts context-aware repulsion strength, which is integrated into a modified similarity computation to penalize alignment with negated semantics, thereby reducing false positive matches. Experiments show that CLIP equipped with CLIPGlasses achieves competitive in-domain performance and outperforms state-of-the-art methods in cross-domain generalization. Its superiority is especially evident under low-resource conditions, indicating stronger robustness across domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21035 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Not Just What's There: Enabling CLIP to Comprehend Negated Visual Descriptions Without Fine-tuning Xiao, Junhao Wu, Zhiyu Lin, Hao Chen, Yi Liu, Yahui Zhao, Xiaoran Wang, Zixu He, Zejiang Computer Vision and Pattern Recognition Multimedia Vision-Language Models (VLMs) like CLIP struggle to understand negation, often embedding affirmatives and negatives similarly (e.g., matching "no dog" with dog images). Existing methods refine negation understanding via fine-tuning CLIP's text encoder, risking overfitting. In this work, we propose CLIPGlasses, a plug-and-play framework that enhances CLIP's ability to comprehend negated visual descriptions. CLIPGlasses adopts a dual-stage design: a Lens module disentangles negated semantics from text embeddings, and a Frame module predicts context-aware repulsion strength, which is integrated into a modified similarity computation to penalize alignment with negated semantics, thereby reducing false positive matches. Experiments show that CLIP equipped with CLIPGlasses achieves competitive in-domain performance and outperforms state-of-the-art methods in cross-domain generalization. Its superiority is especially evident under low-resource conditions, indicating stronger robustness across domains. |
| title | Not Just What's There: Enabling CLIP to Comprehend Negated Visual Descriptions Without Fine-tuning |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2602.21035 |