Connecting the Dots: Surfacing Structure in Documents through AI-Generated Cross-Modal Links
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918344708325376 |
|---|---|
| author | Hwang, Alyssa Kambhamettu, Hita Yang, Yue Patel, Ajay Chang, Joseph Chee Head, Andrew |
| author_facet | Hwang, Alyssa Kambhamettu, Hita Yang, Yue Patel, Ajay Chang, Joseph Chee Head, Andrew |
| contents | Understanding information-dense documents like recipes and scientific papers requires readers to find, interpret, and connect details scattered across text, figures, tables, and other visual elements. These documents are often long and filled with specialized terminology, hindering the ability to locate relevant information or piece together related ideas. Existing tools offer limited support for synthesizing information across media types. As a result, understanding complex material remains cognitively demanding. This paper presents a framework for fine-grained integration of information in complex documents. We instantiate the framework in an augmented reading interface, which populates a scientific paper with clickable points on figures, interactive highlights in the body text, and a persistent reference panel for accessing consolidated details without manual scrolling. In a controlled between-subjects study, we find that participants who read the paper with our tool achieved significantly higher scores on a reading quiz without evidence of increased time to completion or cognitive load. Fine-grained integration provides a systematic way of revealing relationships within a document, supporting engagement with complex, information-dense materials. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_16895 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Connecting the Dots: Surfacing Structure in Documents through AI-Generated Cross-Modal Links Hwang, Alyssa Kambhamettu, Hita Yang, Yue Patel, Ajay Chang, Joseph Chee Head, Andrew Human-Computer Interaction Understanding information-dense documents like recipes and scientific papers requires readers to find, interpret, and connect details scattered across text, figures, tables, and other visual elements. These documents are often long and filled with specialized terminology, hindering the ability to locate relevant information or piece together related ideas. Existing tools offer limited support for synthesizing information across media types. As a result, understanding complex material remains cognitively demanding. This paper presents a framework for fine-grained integration of information in complex documents. We instantiate the framework in an augmented reading interface, which populates a scientific paper with clickable points on figures, interactive highlights in the body text, and a persistent reference panel for accessing consolidated details without manual scrolling. In a controlled between-subjects study, we find that participants who read the paper with our tool achieved significantly higher scores on a reading quiz without evidence of increased time to completion or cognitive load. Fine-grained integration provides a systematic way of revealing relationships within a document, supporting engagement with complex, information-dense materials. |
| title | Connecting the Dots: Surfacing Structure in Documents through AI-Generated Cross-Modal Links |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2602.16895 |