Towards Generalized Source Tracing for Codec-Based Deepfake Speech
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909738304798720 |
|---|---|
| author | Chen, Xuanjun Lin, I-Ming Zhang, Lin Wu, Haibin Lee, Hung-yi Jang, Jyh-Shing Roger |
| author_facet | Chen, Xuanjun Lin, I-Ming Zhang, Lin Wu, Haibin Lee, Hung-yi Jang, Jyh-Shing Roger |
| contents | Recent attempts at source tracing for codec-based deepfake speech (CodecFake), generated by neural audio codec-based speech generation (CoSG) models, have exhibited suboptimal performance. However, how to train source tracing models using simulated CoSG data while maintaining strong performance on real CoSG-generated audio remains an open challenge. In this paper, we show that models trained solely on codec-resynthesized data tend to overfit to non-speech regions and struggle to generalize to unseen content. To mitigate these challenges, we introduce the Semantic-Acoustic Source Tracing Network (SASTNet), which jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding. Our proposed SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating its effectiveness for reliable source tracing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07294 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Generalized Source Tracing for Codec-Based Deepfake Speech Chen, Xuanjun Lin, I-Ming Zhang, Lin Wu, Haibin Lee, Hung-yi Jang, Jyh-Shing Roger Sound Cryptography and Security Machine Learning Audio and Speech Processing Recent attempts at source tracing for codec-based deepfake speech (CodecFake), generated by neural audio codec-based speech generation (CoSG) models, have exhibited suboptimal performance. However, how to train source tracing models using simulated CoSG data while maintaining strong performance on real CoSG-generated audio remains an open challenge. In this paper, we show that models trained solely on codec-resynthesized data tend to overfit to non-speech regions and struggle to generalize to unseen content. To mitigate these challenges, we introduce the Semantic-Acoustic Source Tracing Network (SASTNet), which jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding. Our proposed SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating its effectiveness for reliable source tracing. |
| title | Towards Generalized Source Tracing for Codec-Based Deepfake Speech |
| topic | Sound Cryptography and Security Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2506.07294 |