Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916134988546048 |
|---|---|
| author | Grönquist, Peter Ren, Yufan He, Qingyi Verardo, Alessio Süsstrunk, Sabine |
| author_facet | Grönquist, Peter Ren, Yufan He, Qingyi Verardo, Alessio Süsstrunk, Sabine |
| contents | Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person's expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Some newer methods employ optical flow models to capture this temporal aspect, but they are computationally expensive. In contrast, we propose using the related but often ignored Motion Vectors (MVs) and Information Masks (IMs) from the H.264 video codec, to detect temporal inconsistencies in DeepFakes. Our experiments show that this approach is effective and has minimal computational costs, compared with per-frame RGB-only methods. This could lead to new, real-time temporally-aware DeepFake detection methods for video calls and streaming. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_10788 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors Grönquist, Peter Ren, Yufan He, Qingyi Verardo, Alessio Süsstrunk, Sabine Computer Vision and Pattern Recognition Artificial Intelligence I.5.4; I.4.8; I.2.10; I.4.2 Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person's expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Some newer methods employ optical flow models to capture this temporal aspect, but they are computationally expensive. In contrast, we propose using the related but often ignored Motion Vectors (MVs) and Information Masks (IMs) from the H.264 video codec, to detect temporal inconsistencies in DeepFakes. Our experiments show that this approach is effective and has minimal computational costs, compared with per-frame RGB-only methods. This could lead to new, real-time temporally-aware DeepFake detection methods for video calls and streaming. |
| title | Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence I.5.4; I.4.8; I.2.10; I.4.2 |
| url | https://arxiv.org/abs/2311.10788 |