Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912355890233344 |
|---|---|
| author | Dong, Wenhan Zhao, Yuemeng Sun, Zhen Liu, Yule Peng, Zifan Zheng, Jingyi Zhang, Zongmin Zhang, Ziyi Wu, Jun Wang, Ruiming Xu, Shengmin Huang, Xinyi He, Xinlei |
| author_facet | Dong, Wenhan Zhao, Yuemeng Sun, Zhen Liu, Yule Peng, Zifan Zheng, Jingyi Zhang, Zongmin Zhang, Ziyi Wu, Jun Wang, Ruiming Xu, Shengmin Huang, Xinyi He, Xinlei |
| contents | As large language models (LLMs) are increasingly used in human-centered tasks, assessing their psychological traits is crucial for understanding their social impact and ensuring trustworthy AI alignment. While existing reviews have covered some aspects of related research, several important areas have not been systematically discussed, including detailed discussions of diverse psychological tests, LLM-specific psychological datasets, and the applications of LLMs with psychological traits. To address this gap, we systematically review six key dimensions of applying psychological theories to LLMs: (1) assessment tools; (2) LLM-specific datasets; (3) evaluation metrics (consistency and stability); (4) empirical findings; (5) personality simulation methods; and (6) LLM-based behavior simulation. Our analysis highlights both the strengths and limitations of current methods. While some LLMs exhibit reproducible personality patterns under specific prompting schemes, significant variability remains across tasks and settings. Recognizing methodological challenges such as mismatches between psychological tools and LLMs' capabilities, as well as inconsistencies in evaluation practices, this study aims to propose future directions for developing more interpretable, robust, and generalizable psychological assessment frameworks for LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_00049 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications Dong, Wenhan Zhao, Yuemeng Sun, Zhen Liu, Yule Peng, Zifan Zheng, Jingyi Zhang, Zongmin Zhang, Ziyi Wu, Jun Wang, Ruiming Xu, Shengmin Huang, Xinyi He, Xinlei Computers and Society Computation and Language Human-Computer Interaction Machine Learning As large language models (LLMs) are increasingly used in human-centered tasks, assessing their psychological traits is crucial for understanding their social impact and ensuring trustworthy AI alignment. While existing reviews have covered some aspects of related research, several important areas have not been systematically discussed, including detailed discussions of diverse psychological tests, LLM-specific psychological datasets, and the applications of LLMs with psychological traits. To address this gap, we systematically review six key dimensions of applying psychological theories to LLMs: (1) assessment tools; (2) LLM-specific datasets; (3) evaluation metrics (consistency and stability); (4) empirical findings; (5) personality simulation methods; and (6) LLM-based behavior simulation. Our analysis highlights both the strengths and limitations of current methods. While some LLMs exhibit reproducible personality patterns under specific prompting schemes, significant variability remains across tasks and settings. Recognizing methodological challenges such as mismatches between psychological tools and LLMs' capabilities, as well as inconsistencies in evaluation practices, this study aims to propose future directions for developing more interpretable, robust, and generalizable psychological assessment frameworks for LLMs. |
| title | Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications |
| topic | Computers and Society Computation and Language Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2505.00049 |