GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913308221636608 |
|---|---|
| author | Lu, Hao Niu, Xuesong Wang, Jiyao Wang, Yin Hu, Qingyong Tang, Jiaqi Zhang, Yuting Yuan, Kaishen Huang, Bin Yu, Zitong He, Dengbo Deng, Shuiguang Chen, Hao Chen, Yingcong Shan, Shiguang |
| author_facet | Lu, Hao Niu, Xuesong Wang, Jiyao Wang, Yin Hu, Qingyong Tang, Jiaqi Zhang, Yuting Yuan, Kaishen Huang, Bin Yu, Zitong He, Dengbo Deng, Shuiguang Chen, Hao Chen, Yingcong Shan, Shiguang |
| contents | Multimodal large language models (MLLMs) are designed to process and integrate information from multiple sources, such as text, speech, images, and videos. Despite its success in language understanding, it is critical to evaluate the performance of downstream tasks for better human-centric applications. This paper assesses the application of MLLMs with 5 crucial abilities for affective computing, spanning from visual affective tasks and reasoning tasks. The results show that \gpt has high accuracy in facial action unit recognition and micro-expression detection while its general facial expression recognition performance is not accurate. We also highlight the challenges of achieving fine-grained micro-expression recognition and the potential for further study and demonstrate the versatility and potential of \gpt for handling advanced tasks in emotion recognition and related fields by integrating with task-related agents for more complex tasks, such as heart rate estimation through signal processing. In conclusion, this paper provides valuable insights into the potential applications and challenges of MLLMs in human-centric computing. Our interesting examples are at https://github.com/EnVision-Research/GPT4Affectivity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_05916 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing Lu, Hao Niu, Xuesong Wang, Jiyao Wang, Yin Hu, Qingyong Tang, Jiaqi Zhang, Yuting Yuan, Kaishen Huang, Bin Yu, Zitong He, Dengbo Deng, Shuiguang Chen, Hao Chen, Yingcong Shan, Shiguang Computer Vision and Pattern Recognition Artificial Intelligence Multimodal large language models (MLLMs) are designed to process and integrate information from multiple sources, such as text, speech, images, and videos. Despite its success in language understanding, it is critical to evaluate the performance of downstream tasks for better human-centric applications. This paper assesses the application of MLLMs with 5 crucial abilities for affective computing, spanning from visual affective tasks and reasoning tasks. The results show that \gpt has high accuracy in facial action unit recognition and micro-expression detection while its general facial expression recognition performance is not accurate. We also highlight the challenges of achieving fine-grained micro-expression recognition and the potential for further study and demonstrate the versatility and potential of \gpt for handling advanced tasks in emotion recognition and related fields by integrating with task-related agents for more complex tasks, such as heart rate estimation through signal processing. In conclusion, this paper provides valuable insights into the potential applications and challenges of MLLMs in human-centric computing. Our interesting examples are at https://github.com/EnVision-Research/GPT4Affectivity. |
| title | GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2403.05916 |