Hearing Loss Detection from Facial Expressions in One-on-one Conversations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913198739816448 |
|---|---|
| author | Yin, Yufeng Ananthabhotla, Ishwarya Ithapu, Vamsi Krishna Petridis, Stavros Wu, Yu-Hsiang Miller, Christi |
| author_facet | Yin, Yufeng Ananthabhotla, Ishwarya Ithapu, Vamsi Krishna Petridis, Stavros Wu, Yu-Hsiang Miller, Christi |
| contents | Individuals with impaired hearing experience difficulty in conversations, especially in noisy environments. This difficulty often manifests as a change in behavior and may be captured via facial expressions, such as the expression of discomfort or fatigue. In this work, we build on this idea and introduce the problem of detecting hearing loss from an individual's facial expressions during a conversation. Building machine learning models that can represent hearing-related facial expression changes is a challenge. In addition, models need to disentangle spurious age-related correlations from hearing-driven expressions. To this end, we propose a self-supervised pre-training strategy tailored for the modeling of expression variations. We also use adversarial representation learning to mitigate the age bias. We evaluate our approach on a large-scale egocentric dataset with real-world conversational scenarios involving subjects with hearing loss and show that our method for hearing loss detection achieves superior performance over baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08972 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hearing Loss Detection from Facial Expressions in One-on-one Conversations Yin, Yufeng Ananthabhotla, Ishwarya Ithapu, Vamsi Krishna Petridis, Stavros Wu, Yu-Hsiang Miller, Christi Computer Vision and Pattern Recognition Individuals with impaired hearing experience difficulty in conversations, especially in noisy environments. This difficulty often manifests as a change in behavior and may be captured via facial expressions, such as the expression of discomfort or fatigue. In this work, we build on this idea and introduce the problem of detecting hearing loss from an individual's facial expressions during a conversation. Building machine learning models that can represent hearing-related facial expression changes is a challenge. In addition, models need to disentangle spurious age-related correlations from hearing-driven expressions. To this end, we propose a self-supervised pre-training strategy tailored for the modeling of expression variations. We also use adversarial representation learning to mitigate the age bias. We evaluate our approach on a large-scale egocentric dataset with real-world conversational scenarios involving subjects with hearing loss and show that our method for hearing loss detection achieves superior performance over baselines. |
| title | Hearing Loss Detection from Facial Expressions in One-on-one Conversations |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.08972 |