Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.20822 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909660217344000 |
|---|---|
| author | Myers, Quintin Gao, Yanjun |
| author_facet | Myers, Quintin Gao, Yanjun |
| contents | Large language models (LLMs) are increasingly proposed for detecting and responding to violent content online, yet their ability to reason about morally ambiguous, real-world scenarios remains underexamined. We present the first study to evaluate LLMs using a validated social science instrument designed to measure human response to everyday conflict, namely the Violent Behavior Vignette Questionnaire (VBVQ). To assess potential bias, we introduce persona-based prompting that varies race, age, and geographic identity within the United States. Six LLMs developed across different geopolitical and organizational contexts are evaluated under a unified zero-shot setting. Our study reveals two key findings: (1) LLMs surface-level text generation often diverges from their internal preference for violent responses; (2) their violent tendencies vary across demographics, frequently contradicting established findings in criminology, social science, and psychology. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20822 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes Myers, Quintin Gao, Yanjun Computation and Language Artificial Intelligence Large language models (LLMs) are increasingly proposed for detecting and responding to violent content online, yet their ability to reason about morally ambiguous, real-world scenarios remains underexamined. We present the first study to evaluate LLMs using a validated social science instrument designed to measure human response to everyday conflict, namely the Violent Behavior Vignette Questionnaire (VBVQ). To assess potential bias, we introduce persona-based prompting that varies race, age, and geographic identity within the United States. Six LLMs developed across different geopolitical and organizational contexts are evaluated under a unified zero-shot setting. Our study reveals two key findings: (1) LLMs surface-level text generation often diverges from their internal preference for violent responses; (2) their violent tendencies vary across demographics, frequently contradicting established findings in criminology, social science, and psychology. |
| title | Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.20822 |