Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.09929 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912830716903424 |
|---|---|
| author | Li, Haoyang Hou, Nana Hu, Yuchen Yao, Jixun Siniscalchi, Sabato Marco Zhuang, Xuyi Ye, Deheng Yang, Wei Chng, Eng Siong |
| author_facet | Li, Haoyang Hou, Nana Hu, Yuchen Yao, Jixun Siniscalchi, Sabato Marco Zhuang, Xuyi Ye, Deheng Yang, Wei Chng, Eng Siong |
| contents | Language Model (LM)-based speech enhancement (SE) has recently emerged as a promising direction, but existing approaches predominantly rely on token-level likelihood objectives that weakly reflect human perception. This mismatch limits progress, as optimizing signal accuracy does not always improve naturalness or listening comfort. We address this gap by introducing a perceptually aligned LM-based SE approach. Our method applies Direct Preference Optimization (DPO) with UTMOS, a neural MOS predictor, as a proxy for human ratings, directly steering models toward perceptually preferred outputs. This design directly connects model training to perceptual quality and is broadly applicable within LM-based SE frameworks. On the Deep Noise Suppression Challenge 2020 test sets, our approach consistently improves speech quality metrics, achieving relative gains of up to 56%. To our knowledge, this is the first integration of perceptual feedback into LM-based SE and the first application of DPO in the SE domain, establishing a new paradigm for perceptually aligned enhancement with SE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09929 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Aligning Generative Speech Enhancement with Perceptual Feedback Li, Haoyang Hou, Nana Hu, Yuchen Yao, Jixun Siniscalchi, Sabato Marco Zhuang, Xuyi Ye, Deheng Yang, Wei Chng, Eng Siong Audio and Speech Processing Artificial Intelligence Machine Learning Language Model (LM)-based speech enhancement (SE) has recently emerged as a promising direction, but existing approaches predominantly rely on token-level likelihood objectives that weakly reflect human perception. This mismatch limits progress, as optimizing signal accuracy does not always improve naturalness or listening comfort. We address this gap by introducing a perceptually aligned LM-based SE approach. Our method applies Direct Preference Optimization (DPO) with UTMOS, a neural MOS predictor, as a proxy for human ratings, directly steering models toward perceptually preferred outputs. This design directly connects model training to perceptual quality and is broadly applicable within LM-based SE frameworks. On the Deep Noise Suppression Challenge 2020 test sets, our approach consistently improves speech quality metrics, achieving relative gains of up to 56%. To our knowledge, this is the first integration of perceptual feedback into LM-based SE and the first application of DPO in the SE domain, establishing a new paradigm for perceptually aligned enhancement with SE. |
| title | Aligning Generative Speech Enhancement with Perceptual Feedback |
| topic | Audio and Speech Processing Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.09929 |