Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2605.01809 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915975914323968 |
|---|---|
| author | Yang, Xiaoda Zhang, Majun Pan, Changhao Huang, Nick Yuguang, Yang Zhuo, Fan Zhou, Pengfei Zhou, Jin Shan, Sizhe Yang, Shan Yang, Miles You, Yang Zhao, Zhou |
| author_facet | Yang, Xiaoda Zhang, Majun Pan, Changhao Huang, Nick Yuguang, Yang Zhuo, Fan Zhou, Pengfei Zhou, Jin Shan, Sizhe Yang, Shan Yang, Miles You, Yang Zhao, Zhou |
| contents | Unified audio-visual generation is rapidly gaining industrial and creative relevance, enabling applications in virtual production and interactive media. However, when moving from general audio-video synthesis to music-dance co-generation, the task becomes substantially harder: musical rhythm, phrasing, and accents must drive choreographic motion at fine temporal resolution, and such rhythmic coupling is not captured by unimodal metrics or generic audiovisual consistency scores used in current evaluation practice. We introduce TMD-Bench, a benchmark for text-driven music-dance co-generation that assesses systems across unimodal generation quality, instruction adherence, and cross-modal rhythmic alignment. The benchmark integrates computable physical metrics with perceptual multimodal judgments, and is supported by a curated rhythm-aligned music-dance dataset and a fine-grained Music Captioner for structured music semantics. TMD-Bench further reveals that (i) modern commercial audio-visual models, such as Veo 3 and Sora 2, produce high-quality music and video, while rhythmic coupling remains less consistently optimized and leaves room for improvement, and (ii) our unified baseline RhyJAM trained on rhythm-aligned data achieves competitive beat-level synchronization while maintaining competitive unimodal fidelity. This presents prospects for building next-generation music-dance models that explicitly optimize rhythmic and kinetic coherence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_01809 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TMD-Bench: A Multi-Level Evaluation Paradigm for Music-Dance Co-Generation Yang, Xiaoda Zhang, Majun Pan, Changhao Huang, Nick Yuguang, Yang Zhuo, Fan Zhou, Pengfei Zhou, Jin Shan, Sizhe Yang, Shan Yang, Miles You, Yang Zhao, Zhou Sound Artificial Intelligence Unified audio-visual generation is rapidly gaining industrial and creative relevance, enabling applications in virtual production and interactive media. However, when moving from general audio-video synthesis to music-dance co-generation, the task becomes substantially harder: musical rhythm, phrasing, and accents must drive choreographic motion at fine temporal resolution, and such rhythmic coupling is not captured by unimodal metrics or generic audiovisual consistency scores used in current evaluation practice. We introduce TMD-Bench, a benchmark for text-driven music-dance co-generation that assesses systems across unimodal generation quality, instruction adherence, and cross-modal rhythmic alignment. The benchmark integrates computable physical metrics with perceptual multimodal judgments, and is supported by a curated rhythm-aligned music-dance dataset and a fine-grained Music Captioner for structured music semantics. TMD-Bench further reveals that (i) modern commercial audio-visual models, such as Veo 3 and Sora 2, produce high-quality music and video, while rhythmic coupling remains less consistently optimized and leaves room for improvement, and (ii) our unified baseline RhyJAM trained on rhythm-aligned data achieves competitive beat-level synchronization while maintaining competitive unimodal fidelity. This presents prospects for building next-generation music-dance models that explicitly optimize rhythmic and kinetic coherence. |
| title | TMD-Bench: A Multi-Level Evaluation Paradigm for Music-Dance Co-Generation |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2605.01809 |