Leveraging large multimodal models for audio-video deepfake detection: a pilot study
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908855189897216 |
|---|---|
| author | Cao, Songjun Li, Yuqi Luo, Yunpeng Yin, Jianjun Ma, Long |
| author_facet | Cao, Songjun Li, Yuqi Luo, Yunpeng Yin, Jianjun Ma, Long |
| contents | Audio-visual deepfake detection (AVD) is increasingly important as modern generators can fabricate convincing speech and video. Most current multimodal detectors are small, task-specific models: they work well on curated tests but scale poorly and generalize weakly across domains. We introduce AV-LMMDetect, a supervised fine-tuned (SFT) large multimodal model that casts AVD as a prompted yes/no classification - "Is this video real or fake?". Built on Qwen 2.5 Omni, it jointly analyzes audio and visual streams for deepfake detection and is trained in two stages: lightweight LoRA alignment followed by audio-visual encoder full fine-tuning. On FakeAVCeleb and Mavos-DD, AV-LMMDetect matches or surpasses prior methods and sets a new state of the art on Mavos-DD datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23393 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Leveraging large multimodal models for audio-video deepfake detection: a pilot study Cao, Songjun Li, Yuqi Luo, Yunpeng Yin, Jianjun Ma, Long Sound Computer Vision and Pattern Recognition 68T07, 68T45 I.2.7; I.5.4; H.5.1 Audio-visual deepfake detection (AVD) is increasingly important as modern generators can fabricate convincing speech and video. Most current multimodal detectors are small, task-specific models: they work well on curated tests but scale poorly and generalize weakly across domains. We introduce AV-LMMDetect, a supervised fine-tuned (SFT) large multimodal model that casts AVD as a prompted yes/no classification - "Is this video real or fake?". Built on Qwen 2.5 Omni, it jointly analyzes audio and visual streams for deepfake detection and is trained in two stages: lightweight LoRA alignment followed by audio-visual encoder full fine-tuning. On FakeAVCeleb and Mavos-DD, AV-LMMDetect matches or surpasses prior methods and sets a new state of the art on Mavos-DD datasets. |
| title | Leveraging large multimodal models for audio-video deepfake detection: a pilot study |
| topic | Sound Computer Vision and Pattern Recognition 68T07, 68T45 I.2.7; I.5.4; H.5.1 |
| url | https://arxiv.org/abs/2602.23393 |