Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909222690619392 |
|---|---|
| author | Zezario, Ryandhimas E. Chen, Fei Fuh, Chiou-Shann Wang, Hsin-Min Tsao, Yu |
| author_facet | Zezario, Ryandhimas E. Chen, Fei Fuh, Chiou-Shann Wang, Hsin-Min Tsao, Yu |
| contents | Automated speech intelligibility assessment is pivotal for hearing aid (HA) development. In this paper, we present three novel methods to improve intelligibility prediction accuracy and introduce MBI-Net+, an enhanced version of MBI-Net, the top-performing system in the 1st Clarity Prediction Challenge. MBI-Net+ leverages Whisper's embeddings to create cross-domain acoustic features and includes metadata from speech signals by using a classifier that distinguishes different enhancement methods. Furthermore, MBI-Net+ integrates the hearing-aid speech perception index (HASPI) as a supplementary metric into the objective function to further boost prediction performance. Experimental results demonstrate that MBI-Net+ surpasses several intrusive baseline systems and MBI-Net on the Clarity Prediction Challenge 2023 dataset, validating the effectiveness of incorporating Whisper embeddings, speech metadata, and related complementary metrics to improve prediction performance for HA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_09548 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata Zezario, Ryandhimas E. Chen, Fei Fuh, Chiou-Shann Wang, Hsin-Min Tsao, Yu Audio and Speech Processing Machine Learning Sound Automated speech intelligibility assessment is pivotal for hearing aid (HA) development. In this paper, we present three novel methods to improve intelligibility prediction accuracy and introduce MBI-Net+, an enhanced version of MBI-Net, the top-performing system in the 1st Clarity Prediction Challenge. MBI-Net+ leverages Whisper's embeddings to create cross-domain acoustic features and includes metadata from speech signals by using a classifier that distinguishes different enhancement methods. Furthermore, MBI-Net+ integrates the hearing-aid speech perception index (HASPI) as a supplementary metric into the objective function to further boost prediction performance. Experimental results demonstrate that MBI-Net+ surpasses several intrusive baseline systems and MBI-Net on the Clarity Prediction Challenge 2023 dataset, validating the effectiveness of incorporating Whisper embeddings, speech metadata, and related complementary metrics to improve prediction performance for HA. |
| title | Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata |
| topic | Audio and Speech Processing Machine Learning Sound |
| url | https://arxiv.org/abs/2309.09548 |