Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.08972 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913198739816448 |
|---|---|
| author | Yin, Yufeng Ananthabhotla, Ishwarya Ithapu, Vamsi Krishna Petridis, Stavros Wu, Yu-Hsiang Miller, Christi |
| author_facet | Yin, Yufeng Ananthabhotla, Ishwarya Ithapu, Vamsi Krishna Petridis, Stavros Wu, Yu-Hsiang Miller, Christi |
| contents | Individuals with impaired hearing experience difficulty in conversations, especially in noisy environments. This difficulty often manifests as a change in behavior and may be captured via facial expressions, such as the expression of discomfort or fatigue. In this work, we build on this idea and introduce the problem of detecting hearing loss from an individual's facial expressions during a conversation. Building machine learning models that can represent hearing-related facial expression changes is a challenge. In addition, models need to disentangle spurious age-related correlations from hearing-driven expressions. To this end, we propose a self-supervised pre-training strategy tailored for the modeling of expression variations. We also use adversarial representation learning to mitigate the age bias. We evaluate our approach on a large-scale egocentric dataset with real-world conversational scenarios involving subjects with hearing loss and show that our method for hearing loss detection achieves superior performance over baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08972 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hearing Loss Detection from Facial Expressions in One-on-one Conversations Yin, Yufeng Ananthabhotla, Ishwarya Ithapu, Vamsi Krishna Petridis, Stavros Wu, Yu-Hsiang Miller, Christi Computer Vision and Pattern Recognition Individuals with impaired hearing experience difficulty in conversations, especially in noisy environments. This difficulty often manifests as a change in behavior and may be captured via facial expressions, such as the expression of discomfort or fatigue. In this work, we build on this idea and introduce the problem of detecting hearing loss from an individual's facial expressions during a conversation. Building machine learning models that can represent hearing-related facial expression changes is a challenge. In addition, models need to disentangle spurious age-related correlations from hearing-driven expressions. To this end, we propose a self-supervised pre-training strategy tailored for the modeling of expression variations. We also use adversarial representation learning to mitigate the age bias. We evaluate our approach on a large-scale egocentric dataset with real-world conversational scenarios involving subjects with hearing loss and show that our method for hearing loss detection achieves superior performance over baselines. |
| title | Hearing Loss Detection from Facial Expressions in One-on-one Conversations |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.08972 |