Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914778834796544 |
|---|---|
| author | Apostolidis, Konstantinos Abesser, Jakob Cuccovillo, Luca Mezaris, Vasileios |
| author_facet | Apostolidis, Konstantinos Abesser, Jakob Cuccovillo, Luca Mezaris, Vasileios |
| contents | This paper presents a baseline approach and an experimental protocol for a specific content verification problem: detecting discrepancies between the audio and video modalities in multimedia content. We first design and optimize an audio-visual scene classifier, to compare with existing classification baselines that use both modalities. Then, by applying this classifier separately to the audio and the visual modality, we can detect scene-class inconsistencies between them. To facilitate further research and provide a common evaluation platform, we introduce an experimental protocol and a benchmark dataset simulating such inconsistencies. Our approach achieves state-of-the-art results in scene classification and promising outcomes in audio-visual discrepancies detection, highlighting its potential in content verification applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_00384 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol Apostolidis, Konstantinos Abesser, Jakob Cuccovillo, Luca Mezaris, Vasileios Computer Vision and Pattern Recognition Multimedia Sound Audio and Speech Processing This paper presents a baseline approach and an experimental protocol for a specific content verification problem: detecting discrepancies between the audio and video modalities in multimedia content. We first design and optimize an audio-visual scene classifier, to compare with existing classification baselines that use both modalities. Then, by applying this classifier separately to the audio and the visual modality, we can detect scene-class inconsistencies between them. To facilitate further research and provide a common evaluation platform, we introduce an experimental protocol and a benchmark dataset simulating such inconsistencies. Our approach achieves state-of-the-art results in scene classification and promising outcomes in audio-visual discrepancies detection, highlighting its potential in content verification applications. |
| title | Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol |
| topic | Computer Vision and Pattern Recognition Multimedia Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2405.00384 |