Towards Interactive Deepfake Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910769712463872 |
|---|---|
| author | Qin, Lixiong Jiang, Ning Zhang, Yang Qiu, Yuhan Zeng, Dingheng Hu, Jiani Deng, Weihong |
| author_facet | Qin, Lixiong Jiang, Ning Zhang, Yang Qiu, Yuhan Zeng, Dingheng Hu, Jiani Deng, Weihong |
| contents | Existing deepfake analysis methods are primarily based on discriminative models, which significantly limit their application scenarios. This paper aims to explore interactive deepfake analysis by performing instruction tuning on multi-modal large language models (MLLMs). This will face challenges such as the lack of datasets and benchmarks, and low training efficiency. To address these issues, we introduce (1) a GPT-assisted data construction process resulting in an instruction-following dataset called DFA-Instruct, (2) a benchmark named DFA-Bench, designed to comprehensively evaluate the capabilities of MLLMs in deepfake detection, deepfake classification, and artifact description, and (3) construct an interactive deepfake analysis system called DFA-GPT, as a strong baseline for the community, with the Low-Rank Adaptation (LoRA) module. The dataset and code will be made available at https://github.com/lxq1000/DFA-Instruct to facilitate further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_01164 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Interactive Deepfake Analysis Qin, Lixiong Jiang, Ning Zhang, Yang Qiu, Yuhan Zeng, Dingheng Hu, Jiani Deng, Weihong Computer Vision and Pattern Recognition Existing deepfake analysis methods are primarily based on discriminative models, which significantly limit their application scenarios. This paper aims to explore interactive deepfake analysis by performing instruction tuning on multi-modal large language models (MLLMs). This will face challenges such as the lack of datasets and benchmarks, and low training efficiency. To address these issues, we introduce (1) a GPT-assisted data construction process resulting in an instruction-following dataset called DFA-Instruct, (2) a benchmark named DFA-Bench, designed to comprehensively evaluate the capabilities of MLLMs in deepfake detection, deepfake classification, and artifact description, and (3) construct an interactive deepfake analysis system called DFA-GPT, as a strong baseline for the community, with the Low-Rank Adaptation (LoRA) module. The dataset and code will be made available at https://github.com/lxq1000/DFA-Instruct to facilitate further research. |
| title | Towards Interactive Deepfake Analysis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.01164 |