CogBench: A Large Language Model Benchmark for Multilingual Speech-Based Cognitive Impairment Assessment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914097711284224 |
|---|---|
| author | Feng, Rui Luo, Zhiyao Wang, Wei Song, Yuting Liu, Yong Zhu, Tingting Li, Jianqing Wang, Xingyao |
| author_facet | Feng, Rui Luo, Zhiyao Wang, Wei Song, Yuting Liu, Yong Zhu, Tingting Li, Jianqing Wang, Xingyao |
| contents | Automatic assessment of cognitive impairment from spontaneous speech offers a promising, non-invasive avenue for early cognitive screening. However, current approaches often lack generalizability when deployed across different languages and clinical settings, limiting their practical utility. In this study, we propose CogBench, the first benchmark designed to evaluate the cross-lingual and cross-site generalizability of large language models (LLMs) for speech-based cognitive impairment assessment. Using a unified multimodal pipeline, we evaluate model performance on three speech datasets spanning English and Mandarin: ADReSSo, NCMMSC2021-AD, and a newly collected test set, CIR-E. Our results show that conventional deep learning models degrade substantially when transferred across domains. In contrast, LLMs equipped with chain-of-thought prompting demonstrate better adaptability, though their performance remains sensitive to prompt design. Furthermore, we explore lightweight fine-tuning of LLMs via Low-Rank Adaptation (LoRA), which significantly improves generalization in target domains. These findings offer a critical step toward building clinically useful and linguistically robust speech-based cognitive assessment tools. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03360 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CogBench: A Large Language Model Benchmark for Multilingual Speech-Based Cognitive Impairment Assessment Feng, Rui Luo, Zhiyao Wang, Wei Song, Yuting Liu, Yong Zhu, Tingting Li, Jianqing Wang, Xingyao Artificial Intelligence Automatic assessment of cognitive impairment from spontaneous speech offers a promising, non-invasive avenue for early cognitive screening. However, current approaches often lack generalizability when deployed across different languages and clinical settings, limiting their practical utility. In this study, we propose CogBench, the first benchmark designed to evaluate the cross-lingual and cross-site generalizability of large language models (LLMs) for speech-based cognitive impairment assessment. Using a unified multimodal pipeline, we evaluate model performance on three speech datasets spanning English and Mandarin: ADReSSo, NCMMSC2021-AD, and a newly collected test set, CIR-E. Our results show that conventional deep learning models degrade substantially when transferred across domains. In contrast, LLMs equipped with chain-of-thought prompting demonstrate better adaptability, though their performance remains sensitive to prompt design. Furthermore, we explore lightweight fine-tuning of LLMs via Low-Rank Adaptation (LoRA), which significantly improves generalization in target domains. These findings offer a critical step toward building clinically useful and linguistically robust speech-based cognitive assessment tools. |
| title | CogBench: A Large Language Model Benchmark for Multilingual Speech-Based Cognitive Impairment Assessment |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2508.03360 |