Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908516942348288 |
|---|---|
| author | Dowerah, Sandipana Kulkarni, Atharva Kulkarni, Ajinkya Tran, Hoan My Kalda, Joonas Fedorchenko, Artem Fauve, Benoit Lolive, Damien Alumäe, Tanel Doss, Matthew Magimai |
| author_facet | Dowerah, Sandipana Kulkarni, Atharva Kulkarni, Ajinkya Tran, Hoan My Kalda, Joonas Fedorchenko, Artem Fauve, Benoit Lolive, Damien Alumäe, Tanel Doss, Matthew Magimai |
| contents | Parallel to the development of advanced deepfake audio generation, audio deepfake detection has also seen significant progress. However, a standardized and comprehensive benchmark is still missing. To address this, we introduce Speech DeepFake (DF) Arena, the first comprehensive benchmark for audio deepfake detection. Speech DF Arena provides a toolkit to uniformly evaluate detection systems, currently across 14 diverse datasets and attack scenarios, standardized evaluation metrics and protocols for reproducibility and transparency. It also includes a leaderboard to compare and rank the systems to help researchers and developers enhance their reliability and robustness. We include 14 evaluation sets, 12 state-of-the-art open-source and 3 proprietary detection systems. Our study presents many systems exhibiting high EER in out-of-domain scenarios, highlighting the need for extensive cross-domain evaluation. The leaderboard is hosted on Huggingface1 and a toolkit for reproducing results across the listed datasets is available on GitHub. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_02859 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models Dowerah, Sandipana Kulkarni, Atharva Kulkarni, Ajinkya Tran, Hoan My Kalda, Joonas Fedorchenko, Artem Fauve, Benoit Lolive, Damien Alumäe, Tanel Doss, Matthew Magimai Sound Computation and Language Audio and Speech Processing Parallel to the development of advanced deepfake audio generation, audio deepfake detection has also seen significant progress. However, a standardized and comprehensive benchmark is still missing. To address this, we introduce Speech DeepFake (DF) Arena, the first comprehensive benchmark for audio deepfake detection. Speech DF Arena provides a toolkit to uniformly evaluate detection systems, currently across 14 diverse datasets and attack scenarios, standardized evaluation metrics and protocols for reproducibility and transparency. It also includes a leaderboard to compare and rank the systems to help researchers and developers enhance their reliability and robustness. We include 14 evaluation sets, 12 state-of-the-art open-source and 3 proprietary detection systems. Our study presents many systems exhibiting high EER in out-of-domain scenarios, highlighting the need for extensive cross-domain evaluation. The leaderboard is hosted on Huggingface1 and a toolkit for reproducing results across the listed datasets is available on GitHub. |
| title | Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2509.02859 |