Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.29685 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914613173420032 |
|---|---|
| author | Qi, Yunjin Jiang, Zhaojun Wu, Xuan Pan, Hanxi Wang, Yixuan Liu, Yanfang Ji, Xiang Yu, Churu Zheng, Chunyuan Chen, Yingze He, Jie Chen, Liuqing Gao, Zaifeng |
| author_facet | Qi, Yunjin Jiang, Zhaojun Wu, Xuan Pan, Hanxi Wang, Yixuan Liu, Yanfang Ji, Xiang Yu, Churu Zheng, Chunyuan Chen, Yingze He, Jie Chen, Liuqing Gao, Zaifeng |
| contents | As large language models (LLMs) are increasingly applied in social contexts such as emotional companionship and customer service, measuring their social intelligence has become critical to the quality and safety of human-AI interaction. However, existing social intelligence benchmarks lack a unified framework that organizes social abilities into a unified structure, and therefore cannot enable fine-grained diagnosis. To build the first holistic diagnostic evaluation grounded in social theory, we first construct a social intelligence framework through a literature review and multi-stage expert validation guided by psychometric principles. The resulting framework includes 4 categories and 11 dimensions, each further specified by fine-grained capability facets. Building on this framework, we introduce NICE (Norm, Interaction, Cognition, Experience), a diagnostic benchmark of 137 items operationalized through representative Chinese contexts. Across 5 frontier LLMs and a human reference group, models score higher in aggregate accuracy yet show a consistent weakness in Communication, which the framework localizes to 3 specific capability facets: multi-turn communication, nonverbal communication, and synchrony. NICE thus reframes social intelligence evaluation toward theory-grounded diagnosis of socially consequential weaknesses in LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_29685 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | NICE: A Theory-Grounded Diagnostic Benchmark for Social Intelligence of LLMs Qi, Yunjin Jiang, Zhaojun Wu, Xuan Pan, Hanxi Wang, Yixuan Liu, Yanfang Ji, Xiang Yu, Churu Zheng, Chunyuan Chen, Yingze He, Jie Chen, Liuqing Gao, Zaifeng Artificial Intelligence As large language models (LLMs) are increasingly applied in social contexts such as emotional companionship and customer service, measuring their social intelligence has become critical to the quality and safety of human-AI interaction. However, existing social intelligence benchmarks lack a unified framework that organizes social abilities into a unified structure, and therefore cannot enable fine-grained diagnosis. To build the first holistic diagnostic evaluation grounded in social theory, we first construct a social intelligence framework through a literature review and multi-stage expert validation guided by psychometric principles. The resulting framework includes 4 categories and 11 dimensions, each further specified by fine-grained capability facets. Building on this framework, we introduce NICE (Norm, Interaction, Cognition, Experience), a diagnostic benchmark of 137 items operationalized through representative Chinese contexts. Across 5 frontier LLMs and a human reference group, models score higher in aggregate accuracy yet show a consistent weakness in Communication, which the framework localizes to 3 specific capability facets: multi-turn communication, nonverbal communication, and synchrony. NICE thus reframes social intelligence evaluation toward theory-grounded diagnosis of socially consequential weaknesses in LLMs. |
| title | NICE: A Theory-Grounded Diagnostic Benchmark for Social Intelligence of LLMs |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.29685 |