RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908384134955008 |
|---|---|
| author | Slobodkin, Aviv Taitelbaum, Hagai Bitton, Yonatan Gordon, Brian Sokolik, Michal Guetta, Nitzan Bitton Gueta, Almog Rassin, Royi Lischinski, Dani Szpektor, Idan |
| author_facet | Slobodkin, Aviv Taitelbaum, Hagai Bitton, Yonatan Gordon, Brian Sokolik, Michal Guetta, Nitzan Bitton Gueta, Almog Rassin, Royi Lischinski, Dani Szpektor, Idan |
| contents | Subject-driven text-to-image (T2I) generation aims to produce images that align with a given textual description, while preserving the visual identity from a referenced subject image. Despite its broad downstream applicability - ranging from enhanced personalization in image generation to consistent character representation in video rendering - progress in this field is limited by the lack of reliable automatic evaluation. Existing methods either assess only one aspect of the task (i.e., textual alignment or subject preservation), misalign with human judgments, or rely on costly API-based evaluation. To address this gap, we introduce RefVNLI, a cost-effective metric that evaluates both textual alignment and subject preservation in a single run. Trained on a large-scale dataset derived from video-reasoning benchmarks and image perturbations, RefVNLI outperforms or statistically matches existing baselines across multiple benchmarks and subject categories (e.g., \emph{Animal}, \emph{Object}), achieving up to 6.4-point gains in textual alignment and 5.9-point gains in subject preservation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_17502 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation Slobodkin, Aviv Taitelbaum, Hagai Bitton, Yonatan Gordon, Brian Sokolik, Michal Guetta, Nitzan Bitton Gueta, Almog Rassin, Royi Lischinski, Dani Szpektor, Idan Computer Vision and Pattern Recognition Subject-driven text-to-image (T2I) generation aims to produce images that align with a given textual description, while preserving the visual identity from a referenced subject image. Despite its broad downstream applicability - ranging from enhanced personalization in image generation to consistent character representation in video rendering - progress in this field is limited by the lack of reliable automatic evaluation. Existing methods either assess only one aspect of the task (i.e., textual alignment or subject preservation), misalign with human judgments, or rely on costly API-based evaluation. To address this gap, we introduce RefVNLI, a cost-effective metric that evaluates both textual alignment and subject preservation in a single run. Trained on a large-scale dataset derived from video-reasoning benchmarks and image perturbations, RefVNLI outperforms or statistically matches existing baselines across multiple benchmarks and subject categories (e.g., \emph{Animal}, \emph{Object}), achieving up to 6.4-point gains in textual alignment and 5.9-point gains in subject preservation. |
| title | RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.17502 |