Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911257860243456 |
|---|---|
| author | Richings, Jack Leblanc, Margaux Groves, Ian Nockles, Victoria |
| author_facet | Richings, Jack Leblanc, Margaux Groves, Ian Nockles, Victoria |
| contents | The continually advancing quality of deepfake technology exacerbates the threats of disinformation, fraud, and harassment by making maliciously-generated synthetic content increasingly difficult to distinguish from reality. We introduce a simple yet effective two-stage detection method that achieves an AUROC of over 99.8% on contemporary deepfakes. However, this high performance is short-lived. We show that models trained on this data suffer a recall drop of over 30% when evaluated on deepfakes created with generation techniques from just six months later, demonstrating significant decay as threats evolve. Our analysis reveals two key insights for robust detection. Firstly, continued performance requires the ongoing curation of large, diverse datasets. Second, predictive power comes primarily from static, frame-level artifacts, not temporal inconsistencies. The future of effective deepfake detection therefore depends on rapid data collection and the development of advanced frame-level feature detectors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_07009 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Data Richings, Jack Leblanc, Margaux Groves, Ian Nockles, Victoria Computer Vision and Pattern Recognition 68T07, 68T45 The continually advancing quality of deepfake technology exacerbates the threats of disinformation, fraud, and harassment by making maliciously-generated synthetic content increasingly difficult to distinguish from reality. We introduce a simple yet effective two-stage detection method that achieves an AUROC of over 99.8% on contemporary deepfakes. However, this high performance is short-lived. We show that models trained on this data suffer a recall drop of over 30% when evaluated on deepfakes created with generation techniques from just six months later, demonstrating significant decay as threats evolve. Our analysis reveals two key insights for robust detection. Firstly, continued performance requires the ongoing curation of large, diverse datasets. Second, predictive power comes primarily from static, frame-level artifacts, not temporal inconsistencies. The future of effective deepfake detection therefore depends on rapid data collection and the development of advanced frame-level feature detectors. |
| title | Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Data |
| topic | Computer Vision and Pattern Recognition 68T07, 68T45 |
| url | https://arxiv.org/abs/2511.07009 |