Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.02244 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916421802393600 |
|---|---|
| author | Zhang, Qixuan Wang, Zhifeng Zhang, Dylan Niu, Wenjia Caldwell, Sabrina Gedeon, Tom Liu, Yang Qin, Zhenyue |
| author_facet | Zhang, Qixuan Wang, Zhifeng Zhang, Dylan Niu, Wenjia Caldwell, Sabrina Gedeon, Tom Liu, Yang Qin, Zhenyue |
| contents | Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remains largely unexplored and untapped. Traditional methods in VLLMs struggle with spatial localization and often discard valuable global context. To address this problem, we propose a Set-of-Vision prompting (SoV) approach that enhances zero-shot emotion recognition by using spatial information, such as bounding boxes and facial landmarks, to mark targets precisely. SoV improves accuracy in face count and emotion categorization while preserving the enriched image context. Through a battery of experimentation and analysis of recent commercial or open-source VLLMs, we evaluate the SoV model's ability to comprehend facial expressions in natural environments. Our findings demonstrate the effectiveness of integrating spatial visual prompts into VLLMs for improving emotion recognition performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02244 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Visual Prompting in LLMs for Enhancing Emotion Recognition Zhang, Qixuan Wang, Zhifeng Zhang, Dylan Niu, Wenjia Caldwell, Sabrina Gedeon, Tom Liu, Yang Qin, Zhenyue Computer Vision and Pattern Recognition Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remains largely unexplored and untapped. Traditional methods in VLLMs struggle with spatial localization and often discard valuable global context. To address this problem, we propose a Set-of-Vision prompting (SoV) approach that enhances zero-shot emotion recognition by using spatial information, such as bounding boxes and facial landmarks, to mark targets precisely. SoV improves accuracy in face count and emotion categorization while preserving the enriched image context. Through a battery of experimentation and analysis of recent commercial or open-source VLLMs, we evaluate the SoV model's ability to comprehend facial expressions in natural environments. Our findings demonstrate the effectiveness of integrating spatial visual prompts into VLLMs for improving emotion recognition performance. |
| title | Visual Prompting in LLMs for Enhancing Emotion Recognition |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.02244 |