Target Speaker Extraction by Directly Exploiting Contextual Information in the Time-Frequency Domain
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916140836454400 |
|---|---|
| author | Yang, Xue Bao, Changchun Zhou, Jing Chen, Xianhong |
| author_facet | Yang, Xue Bao, Changchun Zhou, Jing Chen, Xianhong |
| contents | In target speaker extraction, many studies rely on the speaker embedding which is obtained from an enrollment of the target speaker and employed as the guidance. However, solely using speaker embedding may not fully utilize the contextual information contained in the enrollment. In this paper, we directly exploit this contextual information in the time-frequency (T-F) domain. Specifically, the T-F representations of the enrollment and the mixed signal are interacted to compute the weighting matrices through an attention mechanism. These weighting matrices reflect the similarity among different frames of the T-F representations and are further employed to obtain the consistent T-F representations of the enrollment. These consistent representations are served as the guidance, allowing for better exploitation of the contextual information. Furthermore, the proposed method achieves the state-of-the-art performance on the benchmark dataset and shows its effectiveness in the complex scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17146 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Target Speaker Extraction by Directly Exploiting Contextual Information in the Time-Frequency Domain Yang, Xue Bao, Changchun Zhou, Jing Chen, Xianhong Audio and Speech Processing In target speaker extraction, many studies rely on the speaker embedding which is obtained from an enrollment of the target speaker and employed as the guidance. However, solely using speaker embedding may not fully utilize the contextual information contained in the enrollment. In this paper, we directly exploit this contextual information in the time-frequency (T-F) domain. Specifically, the T-F representations of the enrollment and the mixed signal are interacted to compute the weighting matrices through an attention mechanism. These weighting matrices reflect the similarity among different frames of the T-F representations and are further employed to obtain the consistent T-F representations of the enrollment. These consistent representations are served as the guidance, allowing for better exploitation of the contextual information. Furthermore, the proposed method achieves the state-of-the-art performance on the benchmark dataset and shows its effectiveness in the complex scenarios. |
| title | Target Speaker Extraction by Directly Exploiting Contextual Information in the Time-Frequency Domain |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.17146 |