Multimodal Benchmarking and Recommendation of Text-to-Image Generation Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918012853944320 |
|---|---|
| author | Wanaskar, Kapil Jena, Gaytri Eirinaki, Magdalini |
| author_facet | Wanaskar, Kapil Jena, Gaytri Eirinaki, Magdalini |
| contents | This work presents an open-source unified benchmarking and evaluation framework for text-to-image generation models, with a particular focus on the impact of metadata augmented prompts. Leveraging the DeepFashion-MultiModal dataset, we assess generated outputs through a comprehensive set of quantitative metrics, including Weighted Score, CLIP (Contrastive Language Image Pre-training)-based similarity, LPIPS (Learned Perceptual Image Patch Similarity), FID (Frechet Inception Distance), and retrieval-based measures, as well as qualitative analysis. Our results demonstrate that structured metadata enrichments greatly enhance visual realism, semantic fidelity, and model robustness across diverse text-to-image architectures. While not a traditional recommender system, our framework enables task-specific recommendations for model selection and prompt design based on evaluation metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_04650 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multimodal Benchmarking and Recommendation of Text-to-Image Generation Models Wanaskar, Kapil Jena, Gaytri Eirinaki, Magdalini Graphics Artificial Intelligence Information Retrieval Machine Learning This work presents an open-source unified benchmarking and evaluation framework for text-to-image generation models, with a particular focus on the impact of metadata augmented prompts. Leveraging the DeepFashion-MultiModal dataset, we assess generated outputs through a comprehensive set of quantitative metrics, including Weighted Score, CLIP (Contrastive Language Image Pre-training)-based similarity, LPIPS (Learned Perceptual Image Patch Similarity), FID (Frechet Inception Distance), and retrieval-based measures, as well as qualitative analysis. Our results demonstrate that structured metadata enrichments greatly enhance visual realism, semantic fidelity, and model robustness across diverse text-to-image architectures. While not a traditional recommender system, our framework enables task-specific recommendations for model selection and prompt design based on evaluation metrics. |
| title | Multimodal Benchmarking and Recommendation of Text-to-Image Generation Models |
| topic | Graphics Artificial Intelligence Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2505.04650 |