Compression Robust Synthetic Speech Detection Using Patched Spectrogram Transformer
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911781561040896 |
|---|---|
| author | Yadav, Amit Kumar Singh Xiang, Ziyue Bhagtani, Kratika Bestagini, Paolo Tubaro, Stefano Delp, Edward J. |
| author_facet | Yadav, Amit Kumar Singh Xiang, Ziyue Bhagtani, Kratika Bestagini, Paolo Tubaro, Stefano Delp, Edward J. |
| contents | Many deep learning synthetic speech generation tools are readily available. The use of synthetic speech has caused financial fraud, impersonation of people, and misinformation to spread. For this reason forensic methods that can detect synthetic speech have been proposed. Existing methods often overfit on one dataset and their performance reduces substantially in practical scenarios such as detecting synthetic speech shared on social platforms. In this paper we propose, Patched Spectrogram Synthetic Speech Detection Transformer (PS3DT), a synthetic speech detector that converts a time domain speech signal to a mel-spectrogram and processes it in patches using a transformer neural network. We evaluate the detection performance of PS3DT on ASVspoof2019 dataset. Our experiments show that PS3DT performs well on ASVspoof2019 dataset compared to other approaches using spectrogram for synthetic speech detection. We also investigate generalization performance of PS3DT on In-the-Wild dataset. PS3DT generalizes well than several existing methods on detecting synthetic speech from an out-of-distribution dataset. We also evaluate robustness of PS3DT to detect telephone quality synthetic speech and synthetic speech shared on social platforms (compressed speech). PS3DT is robust to compression and can detect telephone quality synthetic speech better than several existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14205 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Compression Robust Synthetic Speech Detection Using Patched Spectrogram Transformer Yadav, Amit Kumar Singh Xiang, Ziyue Bhagtani, Kratika Bestagini, Paolo Tubaro, Stefano Delp, Edward J. Sound Computer Vision and Pattern Recognition Machine Learning Audio and Speech Processing Signal Processing Many deep learning synthetic speech generation tools are readily available. The use of synthetic speech has caused financial fraud, impersonation of people, and misinformation to spread. For this reason forensic methods that can detect synthetic speech have been proposed. Existing methods often overfit on one dataset and their performance reduces substantially in practical scenarios such as detecting synthetic speech shared on social platforms. In this paper we propose, Patched Spectrogram Synthetic Speech Detection Transformer (PS3DT), a synthetic speech detector that converts a time domain speech signal to a mel-spectrogram and processes it in patches using a transformer neural network. We evaluate the detection performance of PS3DT on ASVspoof2019 dataset. Our experiments show that PS3DT performs well on ASVspoof2019 dataset compared to other approaches using spectrogram for synthetic speech detection. We also investigate generalization performance of PS3DT on In-the-Wild dataset. PS3DT generalizes well than several existing methods on detecting synthetic speech from an out-of-distribution dataset. We also evaluate robustness of PS3DT to detect telephone quality synthetic speech and synthetic speech shared on social platforms (compressed speech). PS3DT is robust to compression and can detect telephone quality synthetic speech better than several existing methods. |
| title | Compression Robust Synthetic Speech Detection Using Patched Spectrogram Transformer |
| topic | Sound Computer Vision and Pattern Recognition Machine Learning Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2402.14205 |