Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.20441 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917036533219328 |
|---|---|
| author | Yan, Haoyin Liu, Chengwei Xue, Shaofei Liang, Xiaotao Xue, Zheng |
| author_facet | Yan, Haoyin Liu, Chengwei Xue, Shaofei Liang, Xiaotao Xue, Zheng |
| contents | The development of neural audio codecs (NACs) has largely promoted applications of language models (LMs) to speech processing and understanding. However, there lacks the verification on the effectiveness of autoregressive (AR) LMbased models in unifying different sub-tasks of speech enhancement (SE). In this work, we propose UniSE, a unified decoder-only LM-based framework to handle different SE tasks including speech restoration, target speaker extraction and speech separation. It takes input speech features as conditions and generates discrete tokens of the target speech using AR modeling, which facilitates a compatibility between distinct learning patterns of multiple tasks. Experiments on several benchmarks indicate the proposed UniSE can achieve competitive performance compared to discriminative and generative baselines, showing the capacity of LMs in unifying SE tasks. The demo page is available here: https://github.com/hyyan2k/UniSE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_20441 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniSE: A Unified Framework for Decoder-only Autoregressive LM-based Speech Enhancement Yan, Haoyin Liu, Chengwei Xue, Shaofei Liang, Xiaotao Xue, Zheng Sound Artificial Intelligence The development of neural audio codecs (NACs) has largely promoted applications of language models (LMs) to speech processing and understanding. However, there lacks the verification on the effectiveness of autoregressive (AR) LMbased models in unifying different sub-tasks of speech enhancement (SE). In this work, we propose UniSE, a unified decoder-only LM-based framework to handle different SE tasks including speech restoration, target speaker extraction and speech separation. It takes input speech features as conditions and generates discrete tokens of the target speech using AR modeling, which facilitates a compatibility between distinct learning patterns of multiple tasks. Experiments on several benchmarks indicate the proposed UniSE can achieve competitive performance compared to discriminative and generative baselines, showing the capacity of LMs in unifying SE tasks. The demo page is available here: https://github.com/hyyan2k/UniSE. |
| title | UniSE: A Unified Framework for Decoder-only Autoregressive LM-based Speech Enhancement |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2510.20441 |