Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908425778102272 |
|---|---|
| author | Fosco, Camilo Josephs, Emilie Andonian, Alex Oliva, Aude |
| author_facet | Fosco, Camilo Josephs, Emilie Andonian, Alex Oliva, Aude |
| contents | Deepfakes can fuel online misinformation. As deepfakes get harder to recognize with the naked eye, human users become more reliant on deepfake detection models to help them decide whether a video is real or fake. Currently, models yield a prediction for a video's authenticity, but do not integrate a method for alerting a human user. We introduce a framework for amplifying artifacts in deepfake videos to make them more detectable by people. We propose a novel, semi-supervised Artifact Attention module, which is trained on human responses to create attention maps that highlight video artifacts, and magnify them to create a novel visual indicator we call "Deepfake Caricatures". In a user study, we demonstrate that Caricatures greatly increase human detection, across video presentation times and user engagement levels. We also introduce a deepfake detection model that incorporates the Artifact Attention module to increase its accuracy and robustness. Overall, we demonstrate the success of a human-centered approach to designing deepfake mitigation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2206_00535 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines Fosco, Camilo Josephs, Emilie Andonian, Alex Oliva, Aude Computer Vision and Pattern Recognition Human-Computer Interaction Social and Information Networks Deepfakes can fuel online misinformation. As deepfakes get harder to recognize with the naked eye, human users become more reliant on deepfake detection models to help them decide whether a video is real or fake. Currently, models yield a prediction for a video's authenticity, but do not integrate a method for alerting a human user. We introduce a framework for amplifying artifacts in deepfake videos to make them more detectable by people. We propose a novel, semi-supervised Artifact Attention module, which is trained on human responses to create attention maps that highlight video artifacts, and magnify them to create a novel visual indicator we call "Deepfake Caricatures". In a user study, we demonstrate that Caricatures greatly increase human detection, across video presentation times and user engagement levels. We also introduce a deepfake detection model that incorporates the Artifact Attention module to increase its accuracy and robustness. Overall, we demonstrate the success of a human-centered approach to designing deepfake mitigation methods. |
| title | Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines |
| topic | Computer Vision and Pattern Recognition Human-Computer Interaction Social and Information Networks |
| url | https://arxiv.org/abs/2206.00535 |