For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910645042020352 |
|---|---|
| author | Morra, Lia Santangelo, Antonio Basci, Pietro Piano, Luca Garcea, Fabio Lamberti, Fabrizio Leone, Massimo |
| author_facet | Morra, Lia Santangelo, Antonio Basci, Pietro Piano, Luca Garcea, Fabio Lamberti, Fabrizio Leone, Massimo |
| contents | Social networks are creating a digital world in which the cognitive, emotional, and pragmatic value of the imagery of human faces and bodies is arguably changing. However, researchers in the digital humanities are often ill-equipped to study these phenomena at scale. This work presents FRESCO (Face Representation in E-Societies through Computational Observation), a framework designed to explore the socio-cultural implications of images on social media platforms at scale. FRESCO deconstructs images into numerical and categorical variables using state-of-the-art computer vision techniques, aligning with the principles of visual semiotics. The framework analyzes images across three levels: the plastic level, encompassing fundamental visual features like lines and colors; the figurative level, representing specific entities or concepts; and the enunciation level, which focuses particularly on constructing the point of view of the spectator and observer. These levels are analyzed to discern deeper narrative layers within the imagery. Experimental validation confirms the reliability and utility of FRESCO, and we assess its consistency and precision across two public datasets. Subsequently, we introduce the FRESCO score, a metric derived from the framework's output that serves as a reliable measure of similarity in image content. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03268 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives Morra, Lia Santangelo, Antonio Basci, Pietro Piano, Luca Garcea, Fabio Lamberti, Fabrizio Leone, Massimo Computer Vision and Pattern Recognition Social networks are creating a digital world in which the cognitive, emotional, and pragmatic value of the imagery of human faces and bodies is arguably changing. However, researchers in the digital humanities are often ill-equipped to study these phenomena at scale. This work presents FRESCO (Face Representation in E-Societies through Computational Observation), a framework designed to explore the socio-cultural implications of images on social media platforms at scale. FRESCO deconstructs images into numerical and categorical variables using state-of-the-art computer vision techniques, aligning with the principles of visual semiotics. The framework analyzes images across three levels: the plastic level, encompassing fundamental visual features like lines and colors; the figurative level, representing specific entities or concepts; and the enunciation level, which focuses particularly on constructing the point of view of the spectator and observer. These levels are analyzed to discern deeper narrative layers within the imagery. Experimental validation confirms the reliability and utility of FRESCO, and we assess its consistency and precision across two public datasets. Subsequently, we introduce the FRESCO score, a metric derived from the framework's output that serves as a reliable measure of similarity in image content. |
| title | For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.03268 |