Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2509.16589 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918147042312192 |
|---|---|
| author | Wang, Qiongqiong Sailor, Hardik Bhupendra Liu, Tianchi Zhang, Wenyu Huzaifah, Muhammad Lertcheva, Nattadaporn Sun, Shuo Chen, Nancy F. Wu, Jinyang Aw, AiTi |
| author_facet | Wang, Qiongqiong Sailor, Hardik Bhupendra Liu, Tianchi Zhang, Wenyu Huzaifah, Muhammad Lertcheva, Nattadaporn Sun, Shuo Chen, Nancy F. Wu, Jinyang Aw, AiTi |
| contents | Recent speech-LLMs have shown impressive performance in tasks like transcription and translation, yet they remain limited in understanding the paralinguistic aspects of speech crucial for social and emotional intelligence. We propose CP-Bench, a benchmark for evaluating speech-LLMs on contextual paralinguistic reasoning the integration of verbal content with non-verbal cues like emotion and prosody. The benchmark includes two curated question answering (QA) datasets requiring both linguistic and empathetic understanding. We evaluate state-of-the-art speech-LLMs from both open and closed-source models and perform a comprehensive analysis across different question types. The top two models were further analyzed under temperature tuning to understand its effect on this task. Our benchmark reveals a key gap in existing evaluations and offers insights into building more context-aware and emotionally intelligent speech-capable LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_16589 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data Wang, Qiongqiong Sailor, Hardik Bhupendra Liu, Tianchi Zhang, Wenyu Huzaifah, Muhammad Lertcheva, Nattadaporn Sun, Shuo Chen, Nancy F. Wu, Jinyang Aw, AiTi Computation and Language Artificial Intelligence Recent speech-LLMs have shown impressive performance in tasks like transcription and translation, yet they remain limited in understanding the paralinguistic aspects of speech crucial for social and emotional intelligence. We propose CP-Bench, a benchmark for evaluating speech-LLMs on contextual paralinguistic reasoning the integration of verbal content with non-verbal cues like emotion and prosody. The benchmark includes two curated question answering (QA) datasets requiring both linguistic and empathetic understanding. We evaluate state-of-the-art speech-LLMs from both open and closed-source models and perform a comprehensive analysis across different question types. The top two models were further analyzed under temperature tuning to understand its effect on this task. Our benchmark reveals a key gap in existing evaluations and offers insights into building more context-aware and emotionally intelligent speech-capable LLMs. |
| title | Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.16589 |