Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910971175370752 |
|---|---|
| author | Narain, Jaya Kowtha, Vasudha Lea, Colin Tooley, Lauren Yee, Dianna Mitra, Vikramjit Huang, Zifang Marques, Miquel Espi Huang, Jon Avendano, Carlos Ren, Shirley |
| author_facet | Narain, Jaya Kowtha, Vasudha Lea, Colin Tooley, Lauren Yee, Dianna Mitra, Vikramjit Huang, Zifang Marques, Miquel Espi Huang, Jon Avendano, Carlos Ren, Shirley |
| contents | Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21809 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect Narain, Jaya Kowtha, Vasudha Lea, Colin Tooley, Lauren Yee, Dianna Mitra, Vikramjit Huang, Zifang Marques, Miquel Espi Huang, Jon Avendano, Carlos Ren, Shirley Sound Machine Learning Audio and Speech Processing Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks. |
| title | Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.21809 |