MMSD-Net: Towards Multi-modal Stuttering Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911956958445568 |
|---|---|
| author | Nie, Liangyu Kadiri, Sudarsana Reddy Agrawal, Ruchit |
| author_facet | Nie, Liangyu Kadiri, Sudarsana Reddy Agrawal, Ruchit |
| contents | Stuttering is a common speech impediment that is caused by irregular disruptions in speech production, affecting over 70 million people across the world. Standard automatic speech processing tools do not take speech ailments into account and are thereby not able to generate meaningful results when presented with stuttered speech as input. The automatic detection of stuttering is an integral step towards building efficient, context-aware speech processing systems. While previous approaches explore both statistical and neural approaches for stuttering detection, all of these methods are uni-modal in nature. This paper presents MMSD-Net, the first multi-modal neural framework for stuttering detection. Experiments and results demonstrate that incorporating the visual signal significantly aids stuttering detection, and our model yields an improvement of 2-17% in the F1-score over existing state-of-the-art uni-modal approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_11492 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MMSD-Net: Towards Multi-modal Stuttering Detection Nie, Liangyu Kadiri, Sudarsana Reddy Agrawal, Ruchit Sound Computation and Language Multimedia Audio and Speech Processing Stuttering is a common speech impediment that is caused by irregular disruptions in speech production, affecting over 70 million people across the world. Standard automatic speech processing tools do not take speech ailments into account and are thereby not able to generate meaningful results when presented with stuttered speech as input. The automatic detection of stuttering is an integral step towards building efficient, context-aware speech processing systems. While previous approaches explore both statistical and neural approaches for stuttering detection, all of these methods are uni-modal in nature. This paper presents MMSD-Net, the first multi-modal neural framework for stuttering detection. Experiments and results demonstrate that incorporating the visual signal significantly aids stuttering detection, and our model yields an improvement of 2-17% in the F1-score over existing state-of-the-art uni-modal approaches. |
| title | MMSD-Net: Towards Multi-modal Stuttering Detection |
| topic | Sound Computation and Language Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.11492 |