FlowTSE: Target Speaker Extraction with Flow Matching
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918026839851008 |
|---|---|
| author | Navon, Aviv Shamsian, Aviv Segal-Feldman, Yael Glazer, Neta Hetz, Gil Keshet, Joseph |
| author_facet | Navon, Aviv Shamsian, Aviv Segal-Feldman, Yael Glazer, Neta Hetz, Gil Keshet, Joseph |
| contents | Target speaker extraction (TSE) aims to isolate a specific speaker's speech from a mixture using speaker enrollment as a reference. While most existing approaches are discriminative, recent generative methods for TSE achieve strong results. However, generative methods for TSE remain underexplored, with most existing approaches relying on complex pipelines and pretrained components, leading to computational overhead. In this work, we present FlowTSE, a simple yet effective TSE approach based on conditional flow matching. Our model receives an enrollment audio sample and a mixed speech signal, both represented as mel-spectrograms, with the objective of extracting the target speaker's clean speech. Furthermore, for tasks where phase reconstruction is crucial, we propose a novel vocoder conditioned on the complex STFT of the mixed signal, enabling improved phase estimation. Experimental results on standard TSE benchmarks show that FlowTSE matches or outperforms strong baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14465 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FlowTSE: Target Speaker Extraction with Flow Matching Navon, Aviv Shamsian, Aviv Segal-Feldman, Yael Glazer, Neta Hetz, Gil Keshet, Joseph Audio and Speech Processing Machine Learning Sound Target speaker extraction (TSE) aims to isolate a specific speaker's speech from a mixture using speaker enrollment as a reference. While most existing approaches are discriminative, recent generative methods for TSE achieve strong results. However, generative methods for TSE remain underexplored, with most existing approaches relying on complex pipelines and pretrained components, leading to computational overhead. In this work, we present FlowTSE, a simple yet effective TSE approach based on conditional flow matching. Our model receives an enrollment audio sample and a mixed speech signal, both represented as mel-spectrograms, with the objective of extracting the target speaker's clean speech. Furthermore, for tasks where phase reconstruction is crucial, we propose a novel vocoder conditioned on the complex STFT of the mixed signal, enabling improved phase estimation. Experimental results on standard TSE benchmarks show that FlowTSE matches or outperforms strong baselines. |
| title | FlowTSE: Target Speaker Extraction with Flow Matching |
| topic | Audio and Speech Processing Machine Learning Sound |
| url | https://arxiv.org/abs/2505.14465 |