Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.14648 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915295199756288 |
|---|---|
| author | Feng, Tiantian Lee, Jihwan Xu, Anfeng Lee, Yoonjeong Lertpetchpun, Thanathai Shi, Xuan Wang, Helin Thebaud, Thomas Moro-Velazquez, Laureano Byrd, Dani Dehak, Najim Narayanan, Shrikanth |
| author_facet | Feng, Tiantian Lee, Jihwan Xu, Anfeng Lee, Yoonjeong Lertpetchpun, Thanathai Shi, Xuan Wang, Helin Thebaud, Thomas Moro-Velazquez, Laureano Byrd, Dani Dehak, Najim Narayanan, Shrikanth |
| contents | We introduce Vox-Profile, a comprehensive benchmark to characterize rich speaker and speech traits using speech foundation models. Unlike existing works that focus on a single dimension of speaker traits, Vox-Profile provides holistic and multi-dimensional profiles that reflect both static speaker traits (e.g., age, sex, accent) and dynamic speech properties (e.g., emotion, speech flow). This benchmark is grounded in speech science and linguistics, developed with domain experts to accurately index speaker and speech characteristics. We report benchmark experiments using over 15 publicly available speech datasets and several widely used speech foundation models that target various static and dynamic speaker and speech properties. In addition to benchmark experiments, we showcase several downstream applications supported by Vox-Profile. First, we show that Vox-Profile can augment existing speech recognition datasets to analyze ASR performance variability. Vox-Profile is also used as a tool to evaluate the performance of speech generation systems. Finally, we assess the quality of our automated profiles through comparison with human evaluation and show convergent validity. Vox-Profile is publicly available at: https://github.com/tiantiaf0627/vox-profile-release. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14648 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits Feng, Tiantian Lee, Jihwan Xu, Anfeng Lee, Yoonjeong Lertpetchpun, Thanathai Shi, Xuan Wang, Helin Thebaud, Thomas Moro-Velazquez, Laureano Byrd, Dani Dehak, Najim Narayanan, Shrikanth Sound Audio and Speech Processing We introduce Vox-Profile, a comprehensive benchmark to characterize rich speaker and speech traits using speech foundation models. Unlike existing works that focus on a single dimension of speaker traits, Vox-Profile provides holistic and multi-dimensional profiles that reflect both static speaker traits (e.g., age, sex, accent) and dynamic speech properties (e.g., emotion, speech flow). This benchmark is grounded in speech science and linguistics, developed with domain experts to accurately index speaker and speech characteristics. We report benchmark experiments using over 15 publicly available speech datasets and several widely used speech foundation models that target various static and dynamic speaker and speech properties. In addition to benchmark experiments, we showcase several downstream applications supported by Vox-Profile. First, we show that Vox-Profile can augment existing speech recognition datasets to analyze ASR performance variability. Vox-Profile is also used as a tool to evaluate the performance of speech generation systems. Finally, we assess the quality of our automated profiles through comparison with human evaluation and show convergent validity. Vox-Profile is publicly available at: https://github.com/tiantiaf0627/vox-profile-release. |
| title | Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.14648 |