Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915803602878464 |
|---|---|
| author | Matšenas, Andrius Lello, Anet Lees, Tõnis Peep, Hans Tamm, Kim Lilii |
| author_facet | Matšenas, Andrius Lello, Anet Lees, Tõnis Peep, Hans Tamm, Kim Lilii |
| contents | This study validates Large Language Models (LLMs) as a dynamic alternative to questionnaire-based personality assessment. Using a within-subjects experiment (N=33), we compared Big Five personality scores derived from guided LLM conversations against the gold-standard IPIP-50 questionnaire, while also measuring user-perceived accuracy. Results indicate moderate convergent validity (r=0.38-0.58), with Conscientiousness, Openness, and Neuroticism scores statistically equivalent between methods. Agreeableness and Extraversion showed significant differences, suggesting trait-specific calibration is needed. Notably, participants rated LLM-generated profiles as equally accurate as traditional questionnaire results. These findings suggest conversational AI offers a promising new approach to traditional psychometrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15848 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling Matšenas, Andrius Lello, Anet Lees, Tõnis Peep, Hans Tamm, Kim Lilii Computation and Language Artificial Intelligence I.2.4; I.2.1; J.4 This study validates Large Language Models (LLMs) as a dynamic alternative to questionnaire-based personality assessment. Using a within-subjects experiment (N=33), we compared Big Five personality scores derived from guided LLM conversations against the gold-standard IPIP-50 questionnaire, while also measuring user-perceived accuracy. Results indicate moderate convergent validity (r=0.38-0.58), with Conscientiousness, Openness, and Neuroticism scores statistically equivalent between methods. Agreeableness and Extraversion showed significant differences, suggesting trait-specific calibration is needed. Notably, participants rated LLM-generated profiles as equally accurate as traditional questionnaire results. These findings suggest conversational AI offers a promising new approach to traditional psychometrics. |
| title | Can LLMs Assess Personality? Validating Conversational AI for Trait Profiling |
| topic | Computation and Language Artificial Intelligence I.2.4; I.2.1; J.4 |
| url | https://arxiv.org/abs/2602.15848 |