SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911295843860480 |
|---|---|
| author | Wan, Zhen Yang, Chao-Han Huck Yu, Yahan Tian, Jinchuan Li, Sheng Hu, Ke Chen, Zhehuai Watanabe, Shinji Cheng, Fei Chu, Chenhui Kurohashi, Sadao |
| author_facet | Wan, Zhen Yang, Chao-Han Huck Yu, Yahan Tian, Jinchuan Li, Sheng Hu, Ke Chen, Zhehuai Watanabe, Shinji Cheng, Fei Chu, Chenhui Kurohashi, Sadao |
| contents | We introduce Speech-based Intelligence Quotient (SIQ) as a new form of human cognition-inspired evaluation pipeline for voice understanding large language models, LLM Voice, designed to assess their voice understanding ability. Moving beyond popular voice understanding metrics such as word error rate (WER), SIQ examines LLM Voice across three cognitive levels motivated by Bloom's Taxonomy: (1) Remembering (i.e., WER for verbatim accuracy); (2) Understanding (i.e., similarity of LLM's interpretations); and (3) Application (i.e., QA accuracy for simulating downstream tasks). We demonstrate that SIQ not only quantifies voice understanding abilities but also provides unified comparisons between cascaded methods (e.g., ASR LLM) and end-to-end models, identifies annotation errors in existing benchmarks, and detects hallucinations in LLM Voice. Our framework represents a first-of-its-kind intelligence examination that bridges cognitive principles with voice-oriented benchmarks, while exposing overlooked challenges in multi-modal training. Our code and data will be open source to encourage future studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19361 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models Wan, Zhen Yang, Chao-Han Huck Yu, Yahan Tian, Jinchuan Li, Sheng Hu, Ke Chen, Zhehuai Watanabe, Shinji Cheng, Fei Chu, Chenhui Kurohashi, Sadao Computation and Language Artificial Intelligence Symbolic Computation Sound Audio and Speech Processing We introduce Speech-based Intelligence Quotient (SIQ) as a new form of human cognition-inspired evaluation pipeline for voice understanding large language models, LLM Voice, designed to assess their voice understanding ability. Moving beyond popular voice understanding metrics such as word error rate (WER), SIQ examines LLM Voice across three cognitive levels motivated by Bloom's Taxonomy: (1) Remembering (i.e., WER for verbatim accuracy); (2) Understanding (i.e., similarity of LLM's interpretations); and (3) Application (i.e., QA accuracy for simulating downstream tasks). We demonstrate that SIQ not only quantifies voice understanding abilities but also provides unified comparisons between cascaded methods (e.g., ASR LLM) and end-to-end models, identifies annotation errors in existing benchmarks, and detects hallucinations in LLM Voice. Our framework represents a first-of-its-kind intelligence examination that bridges cognitive principles with voice-oriented benchmarks, while exposing overlooked challenges in multi-modal training. Our code and data will be open source to encourage future studies. |
| title | SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models |
| topic | Computation and Language Artificial Intelligence Symbolic Computation Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2507.19361 |