A Computational Framework to Study Hierarchical Processing in Visual Narratives
Fuente:
Wiley Open Access
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Artículo Open Access |
| Publié: |
Wiley
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1867011362733948928 |
|---|---|
| author | Aditya Upadhyayula Neil Cohn |
| author_facet | Aditya Upadhyayula Neil Cohn Aditya Upadhyayula Neil Cohn |
| collection | Wiley Open Access |
| contents | A Computational Framework to Study Hierarchical Processing in Visual Narratives Aditya Upadhyayula Neil Cohn Cognitive Science AbstractTheories of visual narrative comprehension have advocated for a hierarchical grammar‐based comprehension mechanism, but only limited work has investigated this hierarchy. Here, we provide a computational framework inspired by computational psycholinguistics to address hierarchy in visual narratives. The predictions generated by this framework were compared against behavior data to draw inferences about the hierarchical properties of visual narratives. A segmentation task—where participants ranked all possible segmental boundaries—demonstrated that participants’ preferences were predicted by visual narrative grammar. Three kinds of models using surprisal theory—an Earley parser, a hidden Markov model (HMM), and an n‐gram model—were then used to generate segmentation preferences for the same task. Earley parser's preferences were based on a hierarchical grammar with recursion properties, while the HMM and the n‐grams used a flattened grammar for visual narrative comprehension. Given the differences in the mechanics of these models, contrasting their predictions against behavior data could provide crucial insights into understanding the underlying mechanisms of visual narrative comprehension. By investigating grammatical systems outside of language, this research provides new directions to explore the generic makeup of the cognitive structure of mental representations. 10.1111/cogs.70050 http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| doi_str_mv | 10.1111/cogs.70050 |
| format | Artículo Open Access |
| id | wiley_oa_10_1111_cogs_70050 |
| institution | Wiley Open Access |
| license_str_mv | http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| publishDate | 2025 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | A Computational Framework to Study Hierarchical Processing in Visual Narratives Aditya Upadhyayula Neil Cohn Cognitive Science A Computational Framework to Study Hierarchical Processing in Visual Narratives Aditya Upadhyayula Neil Cohn Cognitive Science AbstractTheories of visual narrative comprehension have advocated for a hierarchical grammar‐based comprehension mechanism, but only limited work has investigated this hierarchy. Here, we provide a computational framework inspired by computational psycholinguistics to address hierarchy in visual narratives. The predictions generated by this framework were compared against behavior data to draw inferences about the hierarchical properties of visual narratives. A segmentation task—where participants ranked all possible segmental boundaries—demonstrated that participants’ preferences were predicted by visual narrative grammar. Three kinds of models using surprisal theory—an Earley parser, a hidden Markov model (HMM), and an n‐gram model—were then used to generate segmentation preferences for the same task. Earley parser's preferences were based on a hierarchical grammar with recursion properties, while the HMM and the n‐grams used a flattened grammar for visual narrative comprehension. Given the differences in the mechanics of these models, contrasting their predictions against behavior data could provide crucial insights into understanding the underlying mechanisms of visual narrative comprehension. By investigating grammatical systems outside of language, this research provides new directions to explore the generic makeup of the cognitive structure of mental representations. 10.1111/cogs.70050 http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| title | A Computational Framework to Study Hierarchical Processing in Visual Narratives |
| topic | Cognitive Science |
| url | https://onlinelibrary.wiley.com/doi/10.1111/cogs.70050 |