Attentive AV-FusionNet: Audio-Visual Quality Prediction with Hybrid Attention
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914271726665728 |
|---|---|
| author | Salaj, Ina Biswas, Arijit |
| author_facet | Salaj, Ina Biswas, Arijit |
| contents | We introduce a novel deep learning-based audio-visual quality (AVQ) prediction model that leverages internal features from state-of-the-art unimodal predictors. Unlike prior approaches that rely on simple fusion strategies, our model employs a hybrid representation that combines learned Generative Machine Listener (GML) audio features with hand-crafted Video Multimethod Assessment Fusion (VMAF) video features. Attention mechanisms capture cross-modal interactions and intra-modal relationships, yielding context-aware quality representations. A modality relevance estimator quantifies each modality's contribution per content, potentially enabling adaptive bitrate allocation. Experiments demonstrate improved AVQ prediction accuracy and robustness across diverse content types. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_16994 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Attentive AV-FusionNet: Audio-Visual Quality Prediction with Hybrid Attention Salaj, Ina Biswas, Arijit Audio and Speech Processing Multimedia Image and Video Processing We introduce a novel deep learning-based audio-visual quality (AVQ) prediction model that leverages internal features from state-of-the-art unimodal predictors. Unlike prior approaches that rely on simple fusion strategies, our model employs a hybrid representation that combines learned Generative Machine Listener (GML) audio features with hand-crafted Video Multimethod Assessment Fusion (VMAF) video features. Attention mechanisms capture cross-modal interactions and intra-modal relationships, yielding context-aware quality representations. A modality relevance estimator quantifies each modality's contribution per content, potentially enabling adaptive bitrate allocation. Experiments demonstrate improved AVQ prediction accuracy and robustness across diverse content types. |
| title | Attentive AV-FusionNet: Audio-Visual Quality Prediction with Hybrid Attention |
| topic | Audio and Speech Processing Multimedia Image and Video Processing |
| url | https://arxiv.org/abs/2509.16994 |