Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.14823 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916083042091008 |
|---|---|
| author | Jiang, Yidi Chen, Zhengyang Tao, Ruijie Deng, Liqun Qian, Yanmin Li, Haizhou |
| author_facet | Jiang, Yidi Chen, Zhengyang Tao, Ruijie Deng, Liqun Qian, Yanmin Li, Haizhou |
| contents | We introduce a novel task named `target speech diarization', which seeks to determine `when target event occurred' within an audio signal. We devise a neural architecture called Prompt-driven Target Speech Diarization (PTSD), that works with diverse prompts that specify the target speech events of interest. We train and evaluate PTSD using sim2spk, sim3spk and sim4spk datasets, which are derived from the Librispeech. We show that the proposed framework accurately localizes target speech events. Furthermore, our framework exhibits versatility through its impressive performance in three diarization-related tasks: target speaker voice activity detection, overlapped speech detection and gender diarization. In particular, PTSD achieves comparable performance to specialized models across these tasks on both real and simulated data. This work serves as a reference benchmark and provides valuable insights into prompt-driven target speech processing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_14823 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Prompt-driven Target Speech Diarization Jiang, Yidi Chen, Zhengyang Tao, Ruijie Deng, Liqun Qian, Yanmin Li, Haizhou Audio and Speech Processing Signal Processing We introduce a novel task named `target speech diarization', which seeks to determine `when target event occurred' within an audio signal. We devise a neural architecture called Prompt-driven Target Speech Diarization (PTSD), that works with diverse prompts that specify the target speech events of interest. We train and evaluate PTSD using sim2spk, sim3spk and sim4spk datasets, which are derived from the Librispeech. We show that the proposed framework accurately localizes target speech events. Furthermore, our framework exhibits versatility through its impressive performance in three diarization-related tasks: target speaker voice activity detection, overlapped speech detection and gender diarization. In particular, PTSD achieves comparable performance to specialized models across these tasks on both real and simulated data. This work serves as a reference benchmark and provides valuable insights into prompt-driven target speech processing. |
| title | Prompt-driven Target Speech Diarization |
| topic | Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2310.14823 |