Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911186906251264 |
|---|---|
| author | Fu, Xingyu Liu, Siyi Xu, Yinuo Lu, Pan Hu, Guangqiuse Yang, Tianbo Anantasagar, Taran Shen, Christopher Mao, Yikai Liu, Yuanzhe Shah, Keyush Lee, Chung Un Choi, Yejin Zou, James Roth, Dan Callison-Burch, Chris |
| author_facet | Fu, Xingyu Liu, Siyi Xu, Yinuo Lu, Pan Hu, Guangqiuse Yang, Tianbo Anantasagar, Taran Shen, Christopher Mao, Yikai Liu, Yuanzhe Shah, Keyush Lee, Chung Un Choi, Yejin Zou, James Roth, Dan Callison-Burch, Chris |
| contents | Can humans identify AI-generated (fake) videos and provide grounded reasons? While video generation models have advanced rapidly, a critical dimension -- whether humans can detect deepfake traces within a generated video, i.e., spatiotemporal grounded visual artifacts that reveal a video as machine generated -- has been largely overlooked. We introduce DeeptraceReward, the first fine-grained, spatially- and temporally- aware benchmark that annotates human-perceived fake traces for video generation reward. The dataset comprises 4.3K detailed annotations across 3.3K high-quality generated videos. Each annotation provides a natural-language explanation, pinpoints a bounding-box region containing the perceived trace, and marks precise onset and offset timestamps. We consolidate these annotations into 9 major categories of deepfake traces that lead humans to identify a video as AI-generated, and train multimodal language models (LMs) as reward models to mimic human judgments and localizations. On DeeptraceReward, our 7B reward model outperforms GPT-5 by 34.7% on average across fake clue identification, grounding, and explanation. Interestingly, we observe a consistent difficulty gradient: binary fake v.s. real classification is substantially easier than fine-grained deepfake trace detection; within the latter, performance degrades from natural language explanations (easiest), to spatial grounding, to temporal labeling (hardest). By foregrounding human-perceived deepfake traces, DeeptraceReward provides a rigorous testbed and training signal for socially aware and trustworthy video generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22646 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs Fu, Xingyu Liu, Siyi Xu, Yinuo Lu, Pan Hu, Guangqiuse Yang, Tianbo Anantasagar, Taran Shen, Christopher Mao, Yikai Liu, Yuanzhe Shah, Keyush Lee, Chung Un Choi, Yejin Zou, James Roth, Dan Callison-Burch, Chris Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Can humans identify AI-generated (fake) videos and provide grounded reasons? While video generation models have advanced rapidly, a critical dimension -- whether humans can detect deepfake traces within a generated video, i.e., spatiotemporal grounded visual artifacts that reveal a video as machine generated -- has been largely overlooked. We introduce DeeptraceReward, the first fine-grained, spatially- and temporally- aware benchmark that annotates human-perceived fake traces for video generation reward. The dataset comprises 4.3K detailed annotations across 3.3K high-quality generated videos. Each annotation provides a natural-language explanation, pinpoints a bounding-box region containing the perceived trace, and marks precise onset and offset timestamps. We consolidate these annotations into 9 major categories of deepfake traces that lead humans to identify a video as AI-generated, and train multimodal language models (LMs) as reward models to mimic human judgments and localizations. On DeeptraceReward, our 7B reward model outperforms GPT-5 by 34.7% on average across fake clue identification, grounding, and explanation. Interestingly, we observe a consistent difficulty gradient: binary fake v.s. real classification is substantially easier than fine-grained deepfake trace detection; within the latter, performance degrades from natural language explanations (easiest), to spatial grounding, to temporal labeling (hardest). By foregrounding human-perceived deepfake traces, DeeptraceReward provides a rigorous testbed and training signal for socially aware and trustworthy video generation. |
| title | Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2509.22646 |