Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2510.24693 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914173152133120 |
|---|---|
| author | Liu, Zihan Niu, Zhikang Xiao, Qiuyang Zheng, Zhisheng Yuan, Ruoqi Zang, Yuhang Cao, Yuhang Dong, Xiaoyi Liang, Jianze Chen, Xie Sun, Leilei Lin, Dahua Wang, Jiaqi |
| author_facet | Liu, Zihan Niu, Zhikang Xiao, Qiuyang Zheng, Zhisheng Yuan, Ruoqi Zang, Yuhang Cao, Yuhang Dong, Xiaoyi Liang, Jianze Chen, Xie Sun, Leilei Lin, Dahua Wang, Jiaqi |
| contents | Despite rapid progress in Multi-modal Large Language Models and Large Audio-Language Models, existing audio benchmarks largely test semantics that can be recovered from text captions, masking deficits in fine-grained perceptual reasoning. We formalize audio 4D intelligence that is defined as reasoning over sound dynamics in time and 3D space, and introduce STAR-Bench to measure it. STAR-Bench combines a Foundational Acoustic Perception setting (six attributes under absolute and relative regimes) with a Holistic Spatio-Temporal Reasoning setting that includes segment reordering for continuous and discrete processes and spatial tasks spanning static localization, multi-source relations, and dynamic trajectories. Our data curation pipeline uses two methods to ensure high-quality samples. For foundational tasks, we use procedurally synthesized and physics-simulated audio. For holistic data, we follow a four-stage process that includes human annotation and final selection based on human performance. Unlike prior benchmarks where caption-only answering reduces accuracy slightly, STAR-Bench induces far larger drops (-31.5\% temporal, -35.2\% spatial), evidencing its focus on linguistically hard-to-describe cues. Evaluating 19 models reveals substantial gaps compared with humans and a capability hierarchy: closed-source models are bottlenecked by fine-grained perception, while open-source models lag across perception, knowledge, and reasoning. Our STAR-Bench provides critical insights and a clear path forward for developing future models with a more robust understanding of the physical world. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_24693 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence Liu, Zihan Niu, Zhikang Xiao, Qiuyang Zheng, Zhisheng Yuan, Ruoqi Zang, Yuhang Cao, Yuhang Dong, Xiaoyi Liang, Jianze Chen, Xie Sun, Leilei Lin, Dahua Wang, Jiaqi Sound Computation and Language Audio and Speech Processing Despite rapid progress in Multi-modal Large Language Models and Large Audio-Language Models, existing audio benchmarks largely test semantics that can be recovered from text captions, masking deficits in fine-grained perceptual reasoning. We formalize audio 4D intelligence that is defined as reasoning over sound dynamics in time and 3D space, and introduce STAR-Bench to measure it. STAR-Bench combines a Foundational Acoustic Perception setting (six attributes under absolute and relative regimes) with a Holistic Spatio-Temporal Reasoning setting that includes segment reordering for continuous and discrete processes and spatial tasks spanning static localization, multi-source relations, and dynamic trajectories. Our data curation pipeline uses two methods to ensure high-quality samples. For foundational tasks, we use procedurally synthesized and physics-simulated audio. For holistic data, we follow a four-stage process that includes human annotation and final selection based on human performance. Unlike prior benchmarks where caption-only answering reduces accuracy slightly, STAR-Bench induces far larger drops (-31.5\% temporal, -35.2\% spatial), evidencing its focus on linguistically hard-to-describe cues. Evaluating 19 models reveals substantial gaps compared with humans and a capability hierarchy: closed-source models are bottlenecked by fine-grained perception, while open-source models lag across perception, knowledge, and reasoning. Our STAR-Bench provides critical insights and a clear path forward for developing future models with a more robust understanding of the physical world. |
| title | STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2510.24693 |