Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.15417 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917055859523584 |
|---|---|
| author | Zhang, Junan Yang, Jing Fang, Zihao Wang, Yuancheng Zhang, Zehua Wang, Zhuo Fan, Fan Wu, Zhizheng |
| author_facet | Zhang, Junan Yang, Jing Fang, Zihao Wang, Yuancheng Zhang, Zehua Wang, Zhuo Fan, Fan Wu, Zhizheng |
| contents | We introduce AnyEnhance, a unified generative model for voice enhancement that processes both speech and singing voices. Based on a masked generative model, AnyEnhance is capable of handling both speech and singing voices, supporting a wide range of enhancement tasks including denoising, dereverberation, declipping, super-resolution, and target speaker extraction, all simultaneously and without fine-tuning. AnyEnhance introduces a prompt-guidance mechanism for in-context learning, which allows the model to natively accept a reference speaker's timbre. In this way, it could boost enhancement performance when a reference audio is available and enable the target speaker extraction task without altering the underlying architecture. Moreover, we also introduce a self-critic mechanism into the generative process for masked generative models, yielding higher-quality outputs through iterative self-assessment and refinement. Extensive experiments on various enhancement tasks demonstrate AnyEnhance outperforms existing methods in terms of both objective metrics and subjective listening tests. Demo audios are publicly available at https://amphionspace.github.io/anyenhance. An open-source implementation is provided at https://github.com/viewfinder-annn/anyenhance-v1-ccf-aatc. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15417 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement Zhang, Junan Yang, Jing Fang, Zihao Wang, Yuancheng Zhang, Zehua Wang, Zhuo Fan, Fan Wu, Zhizheng Sound Artificial Intelligence Machine Learning Audio and Speech Processing We introduce AnyEnhance, a unified generative model for voice enhancement that processes both speech and singing voices. Based on a masked generative model, AnyEnhance is capable of handling both speech and singing voices, supporting a wide range of enhancement tasks including denoising, dereverberation, declipping, super-resolution, and target speaker extraction, all simultaneously and without fine-tuning. AnyEnhance introduces a prompt-guidance mechanism for in-context learning, which allows the model to natively accept a reference speaker's timbre. In this way, it could boost enhancement performance when a reference audio is available and enable the target speaker extraction task without altering the underlying architecture. Moreover, we also introduce a self-critic mechanism into the generative process for masked generative models, yielding higher-quality outputs through iterative self-assessment and refinement. Extensive experiments on various enhancement tasks demonstrate AnyEnhance outperforms existing methods in terms of both objective metrics and subjective listening tests. Demo audios are publicly available at https://amphionspace.github.io/anyenhance. An open-source implementation is provided at https://github.com/viewfinder-annn/anyenhance-v1-ccf-aatc. |
| title | AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement |
| topic | Sound Artificial Intelligence Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.15417 |