Large Language Models for Dysfluency Detection in Stuttered Speech
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910490413760512 |
|---|---|
| author | Wagner, Dominik Bayerl, Sebastian P. Baumann, Ilja Riedhammer, Korbinian Nöth, Elmar Bocklet, Tobias |
| author_facet | Wagner, Dominik Bayerl, Sebastian P. Baumann, Ilja Riedhammer, Korbinian Nöth, Elmar Bocklet, Tobias |
| contents | Accurately detecting dysfluencies in spoken language can help to improve the performance of automatic speech and language processing components and support the development of more inclusive speech and language technologies. Inspired by the recent trend towards the deployment of large language models (LLMs) as universal learners and processors of non-lexical inputs, such as audio and video, we approach the task of multi-label dysfluency detection as a language modeling problem. We present hypotheses candidates generated with an automatic speech recognition system and acoustic representations extracted from an audio encoder model to an LLM, and finetune the system to predict dysfluency labels on three datasets containing English and German stuttered speech. The experimental results show that our system effectively combines acoustic and lexical information and achieves competitive results on the multi-label stuttering detection task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_11025 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Models for Dysfluency Detection in Stuttered Speech Wagner, Dominik Bayerl, Sebastian P. Baumann, Ilja Riedhammer, Korbinian Nöth, Elmar Bocklet, Tobias Sound Computation and Language Audio and Speech Processing Accurately detecting dysfluencies in spoken language can help to improve the performance of automatic speech and language processing components and support the development of more inclusive speech and language technologies. Inspired by the recent trend towards the deployment of large language models (LLMs) as universal learners and processors of non-lexical inputs, such as audio and video, we approach the task of multi-label dysfluency detection as a language modeling problem. We present hypotheses candidates generated with an automatic speech recognition system and acoustic representations extracted from an audio encoder model to an LLM, and finetune the system to predict dysfluency labels on three datasets containing English and German stuttered speech. The experimental results show that our system effectively combines acoustic and lexical information and achieves competitive results on the multi-label stuttering detection task. |
| title | Large Language Models for Dysfluency Detection in Stuttered Speech |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.11025 |