VinaBench: Benchmark for Faithful and Consistent Visual Narratives
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912306321948672 |
|---|---|
| author | Gao, Silin Mathew, Sheryl Mi, Li Mamooler, Sepideh Zhao, Mengjie Wakaki, Hiromi Mitsufuji, Yuki Montariol, Syrielle Bosselut, Antoine |
| author_facet | Gao, Silin Mathew, Sheryl Mi, Li Mamooler, Sepideh Zhao, Mengjie Wakaki, Hiromi Mitsufuji, Yuki Montariol, Syrielle Bosselut, Antoine |
| contents | Visual narrative generation transforms textual narratives into sequences of images illustrating the content of the text. However, generating visual narratives that are faithful to the input text and self-consistent across generated images remains an open challenge, due to the lack of knowledge constraints used for planning the stories. In this work, we propose a new benchmark, VinaBench, to address this challenge. Our benchmark annotates the underlying commonsense and discourse constraints in visual narrative samples, offering systematic scaffolds for learning the implicit strategies of visual storytelling. Based on the incorporated narrative constraints, we further propose novel metrics to closely evaluate the consistency of generated narrative images and the alignment of generations with the input textual narrative. Our results across three generative vision models demonstrate that learning with VinaBench's knowledge constraints effectively improves the faithfulness and cohesion of generated visual narratives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_20871 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VinaBench: Benchmark for Faithful and Consistent Visual Narratives Gao, Silin Mathew, Sheryl Mi, Li Mamooler, Sepideh Zhao, Mengjie Wakaki, Hiromi Mitsufuji, Yuki Montariol, Syrielle Bosselut, Antoine Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Visual narrative generation transforms textual narratives into sequences of images illustrating the content of the text. However, generating visual narratives that are faithful to the input text and self-consistent across generated images remains an open challenge, due to the lack of knowledge constraints used for planning the stories. In this work, we propose a new benchmark, VinaBench, to address this challenge. Our benchmark annotates the underlying commonsense and discourse constraints in visual narrative samples, offering systematic scaffolds for learning the implicit strategies of visual storytelling. Based on the incorporated narrative constraints, we further propose novel metrics to closely evaluate the consistency of generated narrative images and the alignment of generations with the input textual narrative. Our results across three generative vision models demonstrate that learning with VinaBench's knowledge constraints effectively improves the faithfulness and cohesion of generated visual narratives. |
| title | VinaBench: Benchmark for Faithful and Consistent Visual Narratives |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2503.20871 |