CLIP-driven Zero-shot Learning with Ambiguous Labels
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912944729620480 |
|---|---|
| author | Fan, Jinfu Li, Jiangnan Yan, Xiaowen Zhong, Xiaohui Lu, Wenpeng Huang, Linqing |
| author_facet | Fan, Jinfu Li, Jiangnan Yan, Xiaowen Zhong, Xiaohui Lu, Wenpeng Huang, Linqing |
| contents | Zero-shot learning (ZSL) aims to recognize unseen classes by leveraging semantic information from seen classes, but most existing methods assume accurate class labels for training instances. However, in real-world scenarios, noise and ambiguous labels can significantly reduce the performance of ZSL. To address this, we propose a new CLIP-driven partial label zero-shot learning (CLIP-PZSL) framework to handle label ambiguity. First, we use CLIP to extract instance and label features. Then, a semantic mining block fuses these features to extract discriminative label embeddings. We also introduce a partial zero-shot loss, which assigns weights to candidate labels based on their relevance to the instance and aligns instance and label embeddings to minimize semantic mismatch. As the training goes on, the ground-truth labels are progressively identified, and the refined labels and label embeddings in turn help improve the semantic alignment of instance and label features. Comprehensive experiments on several datasets demonstrate the advantage of CLIP-PZSL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_05053 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CLIP-driven Zero-shot Learning with Ambiguous Labels Fan, Jinfu Li, Jiangnan Yan, Xiaowen Zhong, Xiaohui Lu, Wenpeng Huang, Linqing Computer Vision and Pattern Recognition Zero-shot learning (ZSL) aims to recognize unseen classes by leveraging semantic information from seen classes, but most existing methods assume accurate class labels for training instances. However, in real-world scenarios, noise and ambiguous labels can significantly reduce the performance of ZSL. To address this, we propose a new CLIP-driven partial label zero-shot learning (CLIP-PZSL) framework to handle label ambiguity. First, we use CLIP to extract instance and label features. Then, a semantic mining block fuses these features to extract discriminative label embeddings. We also introduce a partial zero-shot loss, which assigns weights to candidate labels based on their relevance to the instance and aligns instance and label embeddings to minimize semantic mismatch. As the training goes on, the ground-truth labels are progressively identified, and the refined labels and label embeddings in turn help improve the semantic alignment of instance and label features. Comprehensive experiments on several datasets demonstrate the advantage of CLIP-PZSL. |
| title | CLIP-driven Zero-shot Learning with Ambiguous Labels |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.05053 |