Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929510825328640 |
|---|---|
| author | Zhang, Zeliang Liu, Zhuo Feng, Mingqian Xu, Chenliang |
| author_facet | Zhang, Zeliang Liu, Zhuo Feng, Mingqian Xu, Chenliang |
| contents | CLIP has demonstrated great versatility in adapting to various downstream tasks, such as image editing and generation, visual question answering, and video understanding. However, CLIP-based applications often suffer from misunderstandings regarding user intent, leading to discrepancies between the required number of objects and the actual outputs in image generation tasks. In this work, we empirically investigate the quantity bias in CLIP. By carefully designing different experimental settings and datasets, we comprehensively evaluate CLIP's understanding of quantity from text, image, and cross-modal perspectives. Our experimental results reveal a quantity bias in CLIP embeddings, impacting the reliability of downstream tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15035 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP Zhang, Zeliang Liu, Zhuo Feng, Mingqian Xu, Chenliang Computer Vision and Pattern Recognition Computation and Language CLIP has demonstrated great versatility in adapting to various downstream tasks, such as image editing and generation, visual question answering, and video understanding. However, CLIP-based applications often suffer from misunderstandings regarding user intent, leading to discrepancies between the required number of objects and the actual outputs in image generation tasks. In this work, we empirically investigate the quantity bias in CLIP. By carefully designing different experimental settings and datasets, we comprehensively evaluate CLIP's understanding of quantity from text, image, and cross-modal perspectives. Our experimental results reveal a quantity bias in CLIP embeddings, impacting the reliability of downstream tasks. |
| title | Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2409.15035 |