Media Content Atlas: A Pipeline to Explore and Investigate Multidimensional Media Space using Multimodal LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917995531468800 |
|---|---|
| author | Cerit, Merve Zelikman, Eric Cho, Mu-Jung Robinson, Thomas N. Reeves, Byron Ram, Nilam Haber, Nick |
| author_facet | Cerit, Merve Zelikman, Eric Cho, Mu-Jung Robinson, Thomas N. Reeves, Byron Ram, Nilam Haber, Nick |
| contents | As digital media use continues to evolve and influence various aspects of life, developing flexible and scalable tools to study complex media experiences is essential. This study introduces the Media Content Atlas (MCA), a novel pipeline designed to help researchers investigate large-scale screen data beyond traditional screen-use metrics. Leveraging multimodal large language models (MLLMs), MCA enables moment-by-moment content analysis, content-based clustering, topic modeling, image retrieval, and interactive visualizations. Evaluated on 1.12 million smartphone screenshots continuously captured during screen use from 112 adults over an entire month, MCA facilitates open-ended exploration and hypothesis generation as well as hypothesis-driven investigations at an unprecedented scale. Expert evaluators underscored its usability and potential for research and intervention design, with clustering results rated 96% relevant and descriptions 83% accurate. By bridging methodological possibilities with domain-specific needs, MCA accelerates both inductive and deductive inquiry, presenting new opportunities for media and HCI research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_16323 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Media Content Atlas: A Pipeline to Explore and Investigate Multidimensional Media Space using Multimodal LLMs Cerit, Merve Zelikman, Eric Cho, Mu-Jung Robinson, Thomas N. Reeves, Byron Ram, Nilam Haber, Nick Human-Computer Interaction Social and Information Networks H.5.0; H.5.1; I.2; J.4 As digital media use continues to evolve and influence various aspects of life, developing flexible and scalable tools to study complex media experiences is essential. This study introduces the Media Content Atlas (MCA), a novel pipeline designed to help researchers investigate large-scale screen data beyond traditional screen-use metrics. Leveraging multimodal large language models (MLLMs), MCA enables moment-by-moment content analysis, content-based clustering, topic modeling, image retrieval, and interactive visualizations. Evaluated on 1.12 million smartphone screenshots continuously captured during screen use from 112 adults over an entire month, MCA facilitates open-ended exploration and hypothesis generation as well as hypothesis-driven investigations at an unprecedented scale. Expert evaluators underscored its usability and potential for research and intervention design, with clustering results rated 96% relevant and descriptions 83% accurate. By bridging methodological possibilities with domain-specific needs, MCA accelerates both inductive and deductive inquiry, presenting new opportunities for media and HCI research. |
| title | Media Content Atlas: A Pipeline to Explore and Investigate Multidimensional Media Space using Multimodal LLMs |
| topic | Human-Computer Interaction Social and Information Networks H.5.0; H.5.1; I.2; J.4 |
| url | https://arxiv.org/abs/2504.16323 |