Infusing Acoustic Pause Context into Text-Based Dementia Assessment
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909298307629056 |
|---|---|
| author | Braun, Franziska Bayerl, Sebastian P. Hönig, Florian Lehfeld, Hartmut Hillemacher, Thomas Bocklet, Tobias Riedhammer, Korbinian |
| author_facet | Braun, Franziska Bayerl, Sebastian P. Hönig, Florian Lehfeld, Hartmut Hillemacher, Thomas Bocklet, Tobias Riedhammer, Korbinian |
| contents | Speech pauses, alongside content and structure, offer a valuable and non-invasive biomarker for detecting dementia. This work investigates the use of pause-enriched transcripts in transformer-based language models to differentiate the cognitive states of subjects with no cognitive impairment, mild cognitive impairment, and Alzheimer's dementia based on their speech from a clinical assessment. We address three binary classification tasks: Onset, monitoring, and dementia exclusion. The performance is evaluated through experiments on a German Verbal Fluency Test and a Picture Description Test, comparing the model's effectiveness across different speech production contexts. Starting from a textual baseline, we investigate the effect of incorporation of pause information and acoustic context. We show the test should be chosen depending on the task, and similarly, lexical pause information and acoustic cross-attention contribute differently. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_15188 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Infusing Acoustic Pause Context into Text-Based Dementia Assessment Braun, Franziska Bayerl, Sebastian P. Hönig, Florian Lehfeld, Hartmut Hillemacher, Thomas Bocklet, Tobias Riedhammer, Korbinian Audio and Speech Processing Computation and Language Sound Speech pauses, alongside content and structure, offer a valuable and non-invasive biomarker for detecting dementia. This work investigates the use of pause-enriched transcripts in transformer-based language models to differentiate the cognitive states of subjects with no cognitive impairment, mild cognitive impairment, and Alzheimer's dementia based on their speech from a clinical assessment. We address three binary classification tasks: Onset, monitoring, and dementia exclusion. The performance is evaluated through experiments on a German Verbal Fluency Test and a Picture Description Test, comparing the model's effectiveness across different speech production contexts. Starting from a textual baseline, we investigate the effect of incorporation of pause information and acoustic context. We show the test should be chosen depending on the task, and similarly, lexical pause information and acoustic cross-attention contribute differently. |
| title | Infusing Acoustic Pause Context into Text-Based Dementia Assessment |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2408.15188 |