Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915803602878464 |
|---|---|
| author | Matšenas, Andrius Lello, Anet Lees, Tõnis Peep, Hans Tamm, Kim Lilii |
| author_facet | Matšenas, Andrius Lello, Anet Lees, Tõnis Peep, Hans Tamm, Kim Lilii |
| contents | This study validates Large Language Models (LLMs) as a dynamic alternative to questionnaire-based personality assessment. Using a within-subjects experiment (N=33), we compared Big Five personality scores derived from guided LLM conversations against the gold-standard IPIP-50 questionnaire, while also measuring user-perceived accuracy. Results indicate moderate convergent validity (r=0.38-0.58), with Conscientiousness, Openness, and Neuroticism scores statistically equivalent between methods. Agreeableness and Extraversion showed significant differences, suggesting trait-specific calibration is needed. Notably, participants rated LLM-generated profiles as equally accurate as traditional questionnaire results. These findings suggest conversational AI offers a promising new approach to traditional psychometrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15848 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling Matšenas, Andrius Lello, Anet Lees, Tõnis Peep, Hans Tamm, Kim Lilii Computation and Language Artificial Intelligence I.2.4; I.2.1; J.4 This study validates Large Language Models (LLMs) as a dynamic alternative to questionnaire-based personality assessment. Using a within-subjects experiment (N=33), we compared Big Five personality scores derived from guided LLM conversations against the gold-standard IPIP-50 questionnaire, while also measuring user-perceived accuracy. Results indicate moderate convergent validity (r=0.38-0.58), with Conscientiousness, Openness, and Neuroticism scores statistically equivalent between methods. Agreeableness and Extraversion showed significant differences, suggesting trait-specific calibration is needed. Notably, participants rated LLM-generated profiles as equally accurate as traditional questionnaire results. These findings suggest conversational AI offers a promising new approach to traditional psychometrics. |
| title | Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling |
| topic | Computation and Language Artificial Intelligence I.2.4; I.2.1; J.4 |
| url | https://arxiv.org/abs/2602.15848 |