Cover Image

Music Audio-Visual Question Answering Requires Specialized Multimodal Designs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	You, Wenhao, Diao, Xingjian, Huang, Wenjun, Zhang, Chunhui, Kong, Keyi, Wu, Weiyi, Ma, Chiyu, Ouyang, Zhongyu, Wu, Tingxuan, Cheng, Ming, Vosoughi, Soroush, Gui, Jiang
Format:	Preprint
Published:	2025
Subjects:	Sound Computer Vision and Pattern Recognition Multimedia Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Musical Representations for Music Performance Question Answering
by: Diao, Xingjian, et al.
Published: (2025)

SoundMind: RL-Incentivized Logic Reasoning for Audio-Language Models
by: Diao, Xingjian, et al.
Published: (2025)

Learning Sparsity for Effective and Efficient Music Performance Question Answering
by: Diao, Xingjian, et al.
Published: (2025)

Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understanding
by: Diao, Xingjian, et al.
Published: (2025)

Jamendo-QA: A Large-Scale Music Question Answering Dataset
by: Koh, Junyoung, et al.
Published: (2025)

Learning Audio Concepts from Counterfactual Natural Language
by: Vosoughi, Ali, et al.
Published: (2024)

VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music
by: Shi, Jiatong, et al.
Published: (2024)

SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answering
by: Yang, Zhe, et al.
Published: (2024)

Addressing Emotion Bias in Music Emotion Recognition and Generation with Frechet Audio Distance
by: Li, Yuanchao, et al.
Published: (2024)

MusicSem: A Semantically Rich Language--Audio Dataset of Natural Music Descriptions
by: Salganik, Rebecca, et al.
Published: (2026)

Multi-Domain Audio Question Answering Benchmark Toward Acoustic Content Reasoning
by: Yang, Chao-Han Huck, et al.
Published: (2025)

MART: Learning Hierarchical Music Audio Representations with Part-Whole Transformer
by: Yao, Dong, et al.
Published: (2023)

MusicScore: A Dataset for Music Score Modeling and Generation
by: Lin, Yuheng, et al.
Published: (2024)

SONIQUE: Video Background Music Generation Using Unpaired Audio-Visual Data
by: Zhang, Liqian, et al.
Published: (2024)

MuseAgent-1: Interactive Grounded Multimodal Understanding of Music Scores and Performance Audio
by: Zhao, Qihao, et al.
Published: (2026)

Rhythmic Foley: A Framework For Seamless Audio-Visual Alignment In Video-to-Audio Synthesis
by: Huang, Zhiqi, et al.
Published: (2024)

Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audio
by: Batlle-Roca, Roser, et al.
Published: (2024)

ProtoVQA: An Adaptable Prototypical Framework for Explainable Fine-Grained Visual Question Answering
by: Diao, Xingjian, et al.
Published: (2025)

Controllable Video-to-Music Generation with Multiple Time-Varying Conditions
by: Wu, Junxian, et al.
Published: (2025)

Towards Multilingual Audio-Visual Question Answering
by: Phukan, Orchid Chetia, et al.
Published: (2024)

Speech-to-See: End-to-End Speech-Driven Open-Set Object Detection
by: Lu, Wenhuan, et al.
Published: (2025)

LoVA: Long-form Video-to-Audio Generation
by: Cheng, Xin, et al.
Published: (2024)

MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence
by: You, Fuming, et al.
Published: (2024)

MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
by: Ma, Ziyang, et al.
Published: (2025)

Generative AI for Music and Audio
by: Dong, Hao-Wen
Published: (2024)

Leveraging Pre-Trained Autoencoders for Interpretable Prototype Learning of Music Audio
by: Alonso-Jiménez, Pablo, et al.
Published: (2024)

M$^{2}$UGen: Multi-modal Music Understanding and Generation with the Power of Large Language Models
by: Liu, Shansong, et al.
Published: (2023)

SteerMusic: Enhanced Musical Consistency for Zero-shot Text-guided and Personalized Music Editing
by: Niu, Xinlei, et al.
Published: (2025)

GVMGen: A General Video-to-Music Generation Model with Hierarchical Attentions
by: Zuo, Heda, et al.
Published: (2025)

Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation
by: Zhou, Ziya, et al.
Published: (2024)

MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models
by: Liu, Shansong, et al.
Published: (2024)

RAVSS: Robust Audio-Visual Speech Separation in Multi-Speaker Scenarios with Missing Visual Cues
by: Pan, Tianrui, et al.
Published: (2024)

Flexible Control in Symbolic Music Generation via Musical Metadata
by: Han, Sangjun, et al.
Published: (2024)

Enhancing Expressiveness in Dance Generation via Integrating Frequency and Music Style Information
by: Huang, Qiaochu, et al.
Published: (2024)

CoheDancers: Enhancing Interactive Group Dance Generation through Music-Driven Coherence Decomposition
by: Yang, Kaixing, et al.
Published: (2024)

Audio-Visual Speaker Tracking: Progress, Challenges, and Future Directions
by: Zhao, Jinzheng, et al.
Published: (2023)

A Comprehensive Survey on Generative AI for Video-to-Music Generation
by: Ji, Shulei, et al.
Published: (2025)

A Survey of Foundation Models for Music Understanding
by: Li, Wenjun, et al.
Published: (2024)

Improving BERT for Symbolic Music Understanding Using Token Denoising and Pianoroll Prediction
by: Wang, Jun-You, et al.
Published: (2025)

Iola Walker: A Mobile Footfall Detection System for Music Composition
by: James, William B.
Published: (2025)