ArtCognition: A Multimodal AI Framework for Affective State Sensing from Visual and Kinematic Drawing Cues
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909984280805376 |
|---|---|
| author | Binaei-Haghighi, Behrad Sajadi, Nafiseh Sadat Liviyan, Mehrad Kharazi, Reyhane Akhavan Amirkhani, Fatemeh Bahrak, Behnam |
| author_facet | Binaei-Haghighi, Behrad Sajadi, Nafiseh Sadat Liviyan, Mehrad Kharazi, Reyhane Akhavan Amirkhani, Fatemeh Bahrak, Behnam |
| contents | The objective assessment of human affective and psychological states presents a significant challenge, particularly through non-verbal channels. This paper introduces digital drawing as a rich and underexplored modality for affective sensing. We present a novel multimodal framework, named ArtCognition, for the automated analysis of the House-Tree-Person (HTP) test, a widely used psychological instrument. ArtCognition uniquely fuses two distinct data streams: static visual features from the final artwork, captured by computer vision models, and dynamic behavioral kinematic cues derived from the drawing process itself, such as stroke speed, pauses, and smoothness. To bridge the gap between low-level features and high-level psychological interpretation, we employ a Retrieval-Augmented Generation (RAG) architecture. This grounds the analysis in established psychological knowledge, enhancing explainability and reducing the potential for model hallucination. Our results demonstrate that the fusion of visual and behavioral kinematic cues provides a more nuanced assessment than either modality alone. We show significant correlations between the extracted multimodal features and standardized psychological metrics, validating the framework's potential as a scalable tool to support clinicians. This work contributes a new methodology for non-intrusive affective state assessment and opens new avenues for technology-assisted mental healthcare. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04297 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ArtCognition: A Multimodal AI Framework for Affective State Sensing from Visual and Kinematic Drawing Cues Binaei-Haghighi, Behrad Sajadi, Nafiseh Sadat Liviyan, Mehrad Kharazi, Reyhane Akhavan Amirkhani, Fatemeh Bahrak, Behnam Machine Learning Computer Vision and Pattern Recognition Human-Computer Interaction Information Retrieval The objective assessment of human affective and psychological states presents a significant challenge, particularly through non-verbal channels. This paper introduces digital drawing as a rich and underexplored modality for affective sensing. We present a novel multimodal framework, named ArtCognition, for the automated analysis of the House-Tree-Person (HTP) test, a widely used psychological instrument. ArtCognition uniquely fuses two distinct data streams: static visual features from the final artwork, captured by computer vision models, and dynamic behavioral kinematic cues derived from the drawing process itself, such as stroke speed, pauses, and smoothness. To bridge the gap between low-level features and high-level psychological interpretation, we employ a Retrieval-Augmented Generation (RAG) architecture. This grounds the analysis in established psychological knowledge, enhancing explainability and reducing the potential for model hallucination. Our results demonstrate that the fusion of visual and behavioral kinematic cues provides a more nuanced assessment than either modality alone. We show significant correlations between the extracted multimodal features and standardized psychological metrics, validating the framework's potential as a scalable tool to support clinicians. This work contributes a new methodology for non-intrusive affective state assessment and opens new avenues for technology-assisted mental healthcare. |
| title | ArtCognition: A Multimodal AI Framework for Affective State Sensing from Visual and Kinematic Drawing Cues |
| topic | Machine Learning Computer Vision and Pattern Recognition Human-Computer Interaction Information Retrieval |
| url | https://arxiv.org/abs/2601.04297 |