FungalZSL: Zero-Shot Fungal Classification with Image Captioning Using a Synthetic Data Approach
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910845924016128 |
|---|---|
| author | Rani, Anju Arroyo, Daniel O. Durdevic, Petar |
| author_facet | Rani, Anju Arroyo, Daniel O. Durdevic, Petar |
| contents | The effectiveness of zero-shot classification in large vision-language models (VLMs), such as Contrastive Language-Image Pre-training (CLIP), depends on access to extensive, well-aligned text-image datasets. In this work, we introduce two complementary data sources, one generated by large language models (LLMs) to describe the stages of fungal growth and another comprising a diverse set of synthetic fungi images. These datasets are designed to enhance CLIPs zero-shot classification capabilities for fungi-related tasks. To ensure effective alignment between text and image data, we project them into CLIPs shared representation space, focusing on different fungal growth stages. We generate text using LLaMA3.2 to bridge modality gaps and synthetically create fungi images. Furthermore, we investigate knowledge transfer by comparing text outputs from different LLM techniques to refine classification across growth stages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_19038 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FungalZSL: Zero-Shot Fungal Classification with Image Captioning Using a Synthetic Data Approach Rani, Anju Arroyo, Daniel O. Durdevic, Petar Computer Vision and Pattern Recognition The effectiveness of zero-shot classification in large vision-language models (VLMs), such as Contrastive Language-Image Pre-training (CLIP), depends on access to extensive, well-aligned text-image datasets. In this work, we introduce two complementary data sources, one generated by large language models (LLMs) to describe the stages of fungal growth and another comprising a diverse set of synthetic fungi images. These datasets are designed to enhance CLIPs zero-shot classification capabilities for fungi-related tasks. To ensure effective alignment between text and image data, we project them into CLIPs shared representation space, focusing on different fungal growth stages. We generate text using LLaMA3.2 to bridge modality gaps and synthetically create fungi images. Furthermore, we investigate knowledge transfer by comparing text outputs from different LLM techniques to refine classification across growth stages. |
| title | FungalZSL: Zero-Shot Fungal Classification with Image Captioning Using a Synthetic Data Approach |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.19038 |