Directional Source Separation for Robust Speech Recognition on Smart Glasses
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909648712368128 |
|---|---|
| author | Feng, Tiantian Lin, Ju Huang, Yiteng He, Weipeng Kalgaonkar, Kaustubh Moritz, Niko Wan, Li Lei, Xin Sun, Ming Seide, Frank |
| author_facet | Feng, Tiantian Lin, Ju Huang, Yiteng He, Weipeng Kalgaonkar, Kaustubh Moritz, Niko Wan, Li Lei, Xin Sun, Ming Seide, Frank |
| contents | Modern smart glasses leverage advanced audio sensing and machine learning technologies to offer real-time transcribing and captioning services, considerably enriching human experiences in daily communications. However, such systems frequently encounter challenges related to environmental noises, resulting in degradation to speech recognition and speaker change detection. To improve voice quality, this work investigates directional source separation using the multi-microphone array. We first explore multiple beamformers to assist source separation modeling by strengthening the directional properties of speech signals. In addition to relying on predetermined beamformers, we investigate neural beamforming in multi-channel source separation, demonstrating that automatic learning directional characteristics effectively improves separation quality. We further compare the ASR performance leveraging separated outputs to noisy inputs. Our results show that directional source separation benefits ASR for the wearer but not for the conversation partner. Lastly, we perform the joint training of the directional source separation and ASR model, achieving the best overall ASR performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_10993 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Directional Source Separation for Robust Speech Recognition on Smart Glasses Feng, Tiantian Lin, Ju Huang, Yiteng He, Weipeng Kalgaonkar, Kaustubh Moritz, Niko Wan, Li Lei, Xin Sun, Ming Seide, Frank Sound Human-Computer Interaction Audio and Speech Processing Modern smart glasses leverage advanced audio sensing and machine learning technologies to offer real-time transcribing and captioning services, considerably enriching human experiences in daily communications. However, such systems frequently encounter challenges related to environmental noises, resulting in degradation to speech recognition and speaker change detection. To improve voice quality, this work investigates directional source separation using the multi-microphone array. We first explore multiple beamformers to assist source separation modeling by strengthening the directional properties of speech signals. In addition to relying on predetermined beamformers, we investigate neural beamforming in multi-channel source separation, demonstrating that automatic learning directional characteristics effectively improves separation quality. We further compare the ASR performance leveraging separated outputs to noisy inputs. Our results show that directional source separation benefits ASR for the wearer but not for the conversation partner. Lastly, we perform the joint training of the directional source separation and ASR model, achieving the best overall ASR performance. |
| title | Directional Source Separation for Robust Speech Recognition on Smart Glasses |
| topic | Sound Human-Computer Interaction Audio and Speech Processing |
| url | https://arxiv.org/abs/2309.10993 |