FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910564639309824 |
|---|---|
| author | Xu, Zhongweiyang Aroudi, Ali Tan, Ke Pandey, Ashutosh Lee, Jung-Suk Xu, Buye Nesta, Francesco |
| author_facet | Xu, Zhongweiyang Aroudi, Ali Tan, Ke Pandey, Ashutosh Lee, Jung-Suk Xu, Buye Nesta, Francesco |
| contents | This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_06468 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses Xu, Zhongweiyang Aroudi, Ali Tan, Ke Pandey, Ashutosh Lee, Jung-Suk Xu, Buye Nesta, Francesco Sound Multimedia Audio and Speech Processing Signal Processing This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications. |
| title | FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses |
| topic | Sound Multimedia Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2408.06468 |