OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915889383735296 |
|---|---|
| author | Kim, Seunghee Park, Bumkyu Jung, Kyudan Lee, Joosung Kim, Soyoon Kim, Jeonghoon Kim, Taeuk Jo, Hwiyeol |
| author_facet | Kim, Seunghee Park, Bumkyu Jung, Kyudan Lee, Joosung Kim, Soyoon Kim, Jeonghoon Kim, Taeuk Jo, Hwiyeol |
| contents | Most testbeds for omni-modal models assess multimodal understanding via textual outputs, leaving it unclear whether these models can properly speak their answers. To study this, we introduce OmniACBench, a benchmark for evaluating context-grounded acoustic control in omni-modal models. Given a spoken instruction, a text script, and an image, a model must read the script aloud with an appropriate tone and manner. OmniACBench comprises 3,559 verified instances covering six acoustic features: speech rate, phonation, pronunciation, emotion, global accent, and timbre. Extensive experiments on eight models reveal their limitations in the proposed setting, despite their strong performance on prior textual-output evaluations. Our analyses show that the main bottleneck lies not in processing individual modalities, but in integrating multimodal context for faithful speech generation. Moreover, we identify three common failure modes-weak direct control, failed implicit inference, and failed multimodal grounding-providing insights for developing models that can verbalize responses effectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_23938 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models Kim, Seunghee Park, Bumkyu Jung, Kyudan Lee, Joosung Kim, Soyoon Kim, Jeonghoon Kim, Taeuk Jo, Hwiyeol Computation and Language Most testbeds for omni-modal models assess multimodal understanding via textual outputs, leaving it unclear whether these models can properly speak their answers. To study this, we introduce OmniACBench, a benchmark for evaluating context-grounded acoustic control in omni-modal models. Given a spoken instruction, a text script, and an image, a model must read the script aloud with an appropriate tone and manner. OmniACBench comprises 3,559 verified instances covering six acoustic features: speech rate, phonation, pronunciation, emotion, global accent, and timbre. Extensive experiments on eight models reveal their limitations in the proposed setting, despite their strong performance on prior textual-output evaluations. Our analyses show that the main bottleneck lies not in processing individual modalities, but in integrating multimodal context for faithful speech generation. Moreover, we identify three common failure modes-weak direct control, failed implicit inference, and failed multimodal grounding-providing insights for developing models that can verbalize responses effectively. |
| title | OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.23938 |