Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929760395853824 |
|---|---|
| author | Han, Yue-Hua Huang, Tai-Ming Hua, Kai-Lung Chen, Jun-Cheng |
| author_facet | Han, Yue-Hua Huang, Tai-Ming Hua, Kai-Lung Chen, Jun-Cheng |
| contents | Generative models have enabled the creation of highly realistic facial-synthetic images, raising significant concerns due to their potential for misuse. Despite rapid advancements in the field of deepfake detection, developing efficient approaches to leverage foundation models for improved generalizability to unseen forgery samples remains challenging. To address this challenge, we propose a novel side-network-based decoder that extracts spatial and temporal cues using the CLIP image encoder for generalized video-based Deepfake detection. Additionally, we introduce Facial Component Guidance (FCG) to enhance spatial learning generalizability by encouraging the model to focus on key facial regions. By leveraging the generic features of a vision-language foundation model, our approach demonstrates promising generalizability on challenging Deepfake datasets while also exhibiting superiority in training data efficiency, parameter efficiency, and model robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_05583 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model Han, Yue-Hua Huang, Tai-Ming Hua, Kai-Lung Chen, Jun-Cheng Computer Vision and Pattern Recognition Generative models have enabled the creation of highly realistic facial-synthetic images, raising significant concerns due to their potential for misuse. Despite rapid advancements in the field of deepfake detection, developing efficient approaches to leverage foundation models for improved generalizability to unseen forgery samples remains challenging. To address this challenge, we propose a novel side-network-based decoder that extracts spatial and temporal cues using the CLIP image encoder for generalized video-based Deepfake detection. Additionally, we introduce Facial Component Guidance (FCG) to enhance spatial learning generalizability by encouraging the model to focus on key facial regions. By leveraging the generic features of a vision-language foundation model, our approach demonstrates promising generalizability on challenging Deepfake datasets while also exhibiting superiority in training data efficiency, parameter efficiency, and model robustness. |
| title | Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.05583 |