The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910763209195520 |
|---|---|
| author | Xie, Yuankun Lu, Yi Fu, Ruibo Wen, Zhengqi Wang, Zhiyong Tao, Jianhua Qi, Xin Wang, Xiaopeng Liu, Yukun Cheng, Haonan Ye, Long Sun, Yi |
| author_facet | Xie, Yuankun Lu, Yi Fu, Ruibo Wen, Zhengqi Wang, Zhiyong Tao, Jianhua Qi, Xin Wang, Xiaopeng Liu, Yukun Cheng, Haonan Ye, Long Sun, Yi |
| contents | With the proliferation of Audio Language Model (ALM) based deepfake audio, there is an urgent need for generalized detection methods. ALM-based deepfake audio currently exhibits widespread, high deception, and type versatility, posing a significant challenge to current audio deepfake detection (ADD) models trained solely on vocoded data. To effectively detect ALM-based deepfake audio, we focus on the mechanism of the ALM-based audio generation method, the conversion from neural codec to waveform. We initially constructed the Codecfake dataset, an open-source, large-scale collection comprising over 1 million audio samples in both English and Chinese, focus on ALM-based audio detection. As countermeasure, to achieve universal detection of deepfake audio and tackle domain ascent bias issue of original sharpness aware minimization (SAM), we propose the CSAM strategy to learn a domain balanced and generalized minima. In our experiments, we first demonstrate that ADD model training with the Codecfake dataset can effectively detects ALM-based audio. Furthermore, our proposed generalization countermeasure yields the lowest average equal error rate (EER) of 0.616% across all test conditions compared to baseline models. The dataset and associated code are available online. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_04880 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio Xie, Yuankun Lu, Yi Fu, Ruibo Wen, Zhengqi Wang, Zhiyong Tao, Jianhua Qi, Xin Wang, Xiaopeng Liu, Yukun Cheng, Haonan Ye, Long Sun, Yi Sound Artificial Intelligence Audio and Speech Processing With the proliferation of Audio Language Model (ALM) based deepfake audio, there is an urgent need for generalized detection methods. ALM-based deepfake audio currently exhibits widespread, high deception, and type versatility, posing a significant challenge to current audio deepfake detection (ADD) models trained solely on vocoded data. To effectively detect ALM-based deepfake audio, we focus on the mechanism of the ALM-based audio generation method, the conversion from neural codec to waveform. We initially constructed the Codecfake dataset, an open-source, large-scale collection comprising over 1 million audio samples in both English and Chinese, focus on ALM-based audio detection. As countermeasure, to achieve universal detection of deepfake audio and tackle domain ascent bias issue of original sharpness aware minimization (SAM), we propose the CSAM strategy to learn a domain balanced and generalized minima. In our experiments, we first demonstrate that ADD model training with the Codecfake dataset can effectively detects ALM-based audio. Furthermore, our proposed generalization countermeasure yields the lowest average equal error rate (EER) of 0.616% across all test conditions compared to baseline models. The dataset and associated code are available online. |
| title | The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2405.04880 |