Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916473023234048 |
|---|---|
| author | Cheng, Sheng Patel, Maitreya Yang, Yezhou |
| author_facet | Cheng, Sheng Patel, Maitreya Yang, Yezhou |
| contents | Despite advancements in text-to-image models, generating images that precisely align with textual descriptions remains challenging due to misalignment in training data. In this paper, we analyze the critical role of caption precision and recall in text-to-image model training. Our analysis of human-annotated captions shows that both precision and recall are important for text-image alignment, but precision has a more significant impact. Leveraging these insights, we utilize Large Vision Language Models to generate synthetic captions for training. Models trained with these synthetic captions show similar behavior to those trained on human-annotated captions, underscores the potential for synthetic data in text-to-image training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_05079 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model Cheng, Sheng Patel, Maitreya Yang, Yezhou Computer Vision and Pattern Recognition Computation and Language Despite advancements in text-to-image models, generating images that precisely align with textual descriptions remains challenging due to misalignment in training data. In this paper, we analyze the critical role of caption precision and recall in text-to-image model training. Our analysis of human-annotated captions shows that both precision and recall are important for text-image alignment, but precision has a more significant impact. Leveraging these insights, we utilize Large Vision Language Models to generate synthetic captions for training. Models trained with these synthetic captions show similar behavior to those trained on human-annotated captions, underscores the potential for synthetic data in text-to-image training. |
| title | Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2411.05079 |