TaxaAdapter: Vision Taxonomy Models are Key to Fine-grained Image Generation over the Tree of Life
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908916827291648 |
|---|---|
| author | Khurana, Mridul Monsefi, Amin Karimi Lee, Justin Sawhney, Medha Carlyn, David Chae, Julia Gu, Jianyang Ramnath, Rajiv Beery, Sara Chao, Wei-Lun Karpatne, Anuj Zhang, Cheng |
| author_facet | Khurana, Mridul Monsefi, Amin Karimi Lee, Justin Sawhney, Medha Carlyn, David Chae, Julia Gu, Jianyang Ramnath, Rajiv Beery, Sara Chao, Wei-Lun Karpatne, Anuj Zhang, Cheng |
| contents | Accurately generating images across the Tree of Life is difficult: there are over 10M distinct species on Earth, many of which differ only by subtle visual traits. Despite the remarkable progress in text-to-image synthesis, existing models often fail to capture the fine-grained visual cues that define species identity, even when their outputs appear photo-realistic. To this end, we propose TaxaAdapter, a simple and lightweight approach that incorporates Vision Taxonomy Models (VTMs) such as BioCLIP to guide fine-grained species generation. Our method injects VTM embeddings into a frozen text-to-image diffusion model, improving species-level fidelity while preserving flexible text control over attributes such as pose, style, and background. Extensive experiments demonstrate that TaxaAdapter consistently improves morphology fidelity and species-identity accuracy over strong baselines, with a cleaner architecture and training recipe. To better evaluate these improvements, we also introduce a multimodal Large Language Model-based metric that summarizes trait-level descriptions from generated and real images, providing a more interpretable measure of morphological consistency. Beyond this, we observe that TaxaAdapter exhibits strong generalization capabilities, enabling species synthesis in challenging regimes such as few-shot species with only a handful of training images and even species unseen during training. Overall, our results highlight that VTMs are a key ingredient for scalable, fine-grained species generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_26128 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TaxaAdapter: Vision Taxonomy Models are Key to Fine-grained Image Generation over the Tree of Life Khurana, Mridul Monsefi, Amin Karimi Lee, Justin Sawhney, Medha Carlyn, David Chae, Julia Gu, Jianyang Ramnath, Rajiv Beery, Sara Chao, Wei-Lun Karpatne, Anuj Zhang, Cheng Computer Vision and Pattern Recognition Accurately generating images across the Tree of Life is difficult: there are over 10M distinct species on Earth, many of which differ only by subtle visual traits. Despite the remarkable progress in text-to-image synthesis, existing models often fail to capture the fine-grained visual cues that define species identity, even when their outputs appear photo-realistic. To this end, we propose TaxaAdapter, a simple and lightweight approach that incorporates Vision Taxonomy Models (VTMs) such as BioCLIP to guide fine-grained species generation. Our method injects VTM embeddings into a frozen text-to-image diffusion model, improving species-level fidelity while preserving flexible text control over attributes such as pose, style, and background. Extensive experiments demonstrate that TaxaAdapter consistently improves morphology fidelity and species-identity accuracy over strong baselines, with a cleaner architecture and training recipe. To better evaluate these improvements, we also introduce a multimodal Large Language Model-based metric that summarizes trait-level descriptions from generated and real images, providing a more interpretable measure of morphological consistency. Beyond this, we observe that TaxaAdapter exhibits strong generalization capabilities, enabling species synthesis in challenging regimes such as few-shot species with only a handful of training images and even species unseen during training. Overall, our results highlight that VTMs are a key ingredient for scalable, fine-grained species generation. |
| title | TaxaAdapter: Vision Taxonomy Models are Key to Fine-grained Image Generation over the Tree of Life |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.26128 |