Improving Resource-Efficient Speech Enhancement via Neural Differentiable DSP Vocoder Refinement
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908496396550144 |
|---|---|
| author | Guimarães, Heitor R. Tan, Ke Azcarreta, Juan Alvarez, Jesus Agrawal, Prabhav Pandey, Ashutosh Xu, Buye |
| author_facet | Guimarães, Heitor R. Tan, Ke Azcarreta, Juan Alvarez, Jesus Agrawal, Prabhav Pandey, Ashutosh Xu, Buye |
| contents | Deploying speech enhancement (SE) systems in wearable devices, such as smart glasses, is challenging due to the limited computational resources on the device. Although deep learning methods have achieved high-quality results, their computational cost limits their feasibility on embedded platforms. This work presents an efficient end-to-end SE framework that leverages a Differentiable Digital Signal Processing (DDSP) vocoder for high-quality speech synthesis. First, a compact neural network predicts enhanced acoustic features from noisy speech: spectral envelope, fundamental frequency (F0), and periodicity. These features are fed into the DDSP vocoder to synthesize the enhanced waveform. The system is trained end-to-end with STFT and adversarial losses, enabling direct optimization at the feature and waveform levels. Experimental results show that our method improves intelligibility and quality by 4% (STOI) and 19% (DNSMOS) over strong baselines without significantly increasing computation, making it well-suited for real-time applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_14709 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Resource-Efficient Speech Enhancement via Neural Differentiable DSP Vocoder Refinement Guimarães, Heitor R. Tan, Ke Azcarreta, Juan Alvarez, Jesus Agrawal, Prabhav Pandey, Ashutosh Xu, Buye Audio and Speech Processing Sound Deploying speech enhancement (SE) systems in wearable devices, such as smart glasses, is challenging due to the limited computational resources on the device. Although deep learning methods have achieved high-quality results, their computational cost limits their feasibility on embedded platforms. This work presents an efficient end-to-end SE framework that leverages a Differentiable Digital Signal Processing (DDSP) vocoder for high-quality speech synthesis. First, a compact neural network predicts enhanced acoustic features from noisy speech: spectral envelope, fundamental frequency (F0), and periodicity. These features are fed into the DDSP vocoder to synthesize the enhanced waveform. The system is trained end-to-end with STFT and adversarial losses, enabling direct optimization at the feature and waveform levels. Experimental results show that our method improves intelligibility and quality by 4% (STOI) and 19% (DNSMOS) over strong baselines without significantly increasing computation, making it well-suited for real-time applications. |
| title | Improving Resource-Efficient Speech Enhancement via Neural Differentiable DSP Vocoder Refinement |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2508.14709 |