Predicting Cognitive Decline: A Multimodal AI Approach to Dementia Screening from Speech
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912230885294080 |
|---|---|
| author | Chi, Lei Sharma, Arav Gebhardt, Ari Colonel, Joseph T. |
| author_facet | Chi, Lei Sharma, Arav Gebhardt, Ari Colonel, Joseph T. |
| contents | Recent progress has been made in detecting early stage dementia entirely through recordings of patient speech. Multimodal speech analysis methods were applied to the PROCESS challenge, which requires participants to use audio recordings of clinical interviews to predict patients as healthy control, mild cognitive impairment (MCI), or dementia and regress the patient's Mini-Mental State Exam (MMSE) scores. The approach implemented in this work combines acoustic features (eGeMAPS and Prosody) with embeddings from Whisper and RoBERTa models, achieving competitive results in both regression (RMSE: 2.7666) and classification (Macro-F1 score: 0.5774) tasks. Additionally, a novel two-tiered classification setup is utilized to better differentiate between MCI and dementia. Our approach achieved strong results on the test set, ranking seventh on regression and eleventh on classification out of thirty-seven teams, exceeding the baseline results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_08862 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Predicting Cognitive Decline: A Multimodal AI Approach to Dementia Screening from Speech Chi, Lei Sharma, Arav Gebhardt, Ari Colonel, Joseph T. Audio and Speech Processing Recent progress has been made in detecting early stage dementia entirely through recordings of patient speech. Multimodal speech analysis methods were applied to the PROCESS challenge, which requires participants to use audio recordings of clinical interviews to predict patients as healthy control, mild cognitive impairment (MCI), or dementia and regress the patient's Mini-Mental State Exam (MMSE) scores. The approach implemented in this work combines acoustic features (eGeMAPS and Prosody) with embeddings from Whisper and RoBERTa models, achieving competitive results in both regression (RMSE: 2.7666) and classification (Macro-F1 score: 0.5774) tasks. Additionally, a novel two-tiered classification setup is utilized to better differentiate between MCI and dementia. Our approach achieved strong results on the test set, ranking seventh on regression and eleventh on classification out of thirty-seven teams, exceeding the baseline results. |
| title | Predicting Cognitive Decline: A Multimodal AI Approach to Dementia Screening from Speech |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.08862 |