Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917363142623232 |
|---|---|
| author | Ilinykh, Nikolai Jang, Hyewon Lappin, Shalom Sayeed, Asad Loáiciga, Sharid |
| author_facet | Ilinykh, Nikolai Jang, Hyewon Lappin, Shalom Sayeed, Asad Loáiciga, Sharid |
| contents | We study narrative coherence in visually grounded stories by comparing human-written narratives with those generated by vision-language models (VLMs) on the Visual Writing Prompts corpus. Using a set of metrics that capture different aspects of narrative coherence, including coreference, discourse relation types, topic continuity, character persistence, and multimodal character grounding, we compute a narrative coherence score. We find that VLMs show broadly similar coherence profiles that differ systematically from those of humans. In addition, differences for individual measures are often subtle, but they become clearer when considered jointly. Overall, our results indicate that, despite human-like surface fluency, model narratives exhibit systematic differences from those of humans in how they organise discourse across a visually grounded story. Our code is available at https://github.com/GU-CLASP/coherence-driven-humans. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_25537 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence Ilinykh, Nikolai Jang, Hyewon Lappin, Shalom Sayeed, Asad Loáiciga, Sharid Computation and Language We study narrative coherence in visually grounded stories by comparing human-written narratives with those generated by vision-language models (VLMs) on the Visual Writing Prompts corpus. Using a set of metrics that capture different aspects of narrative coherence, including coreference, discourse relation types, topic continuity, character persistence, and multimodal character grounding, we compute a narrative coherence score. We find that VLMs show broadly similar coherence profiles that differ systematically from those of humans. In addition, differences for individual measures are often subtle, but they become clearer when considered jointly. Overall, our results indicate that, despite human-like surface fluency, model narratives exhibit systematic differences from those of humans in how they organise discourse across a visually grounded story. Our code is available at https://github.com/GU-CLASP/coherence-driven-humans. |
| title | Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.25537 |