Cover Image

Do Psychometric Tests Work for Large Language Models? Evaluation of Tests on Sexism, Racism, and Morality

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jung, Jana, Lutz, Marlene, Sen, Indira, Strohmaier, Markus
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Models
by: Lutz, Marlene, et al.
Published: (2025)

Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models
by: Kreutner, Maximilian, et al.
Published: (2025)

Local Contrastive Editing of Gender Stereotypes
by: Lutz, Marlene, et al.
Published: (2024)

Tell Me What You Know About Sexism: Expert-LLM Interaction Strategies and Co-Created Definitions for Zero-Shot Sexism Detection
by: Reuver, Myrthe, et al.
Published: (2025)

Neural network embeddings recover value dimensions from psychometric survey items on par with human data
by: Pellert, Max, et al.
Published: (2025)

Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs
by: Sen, Indira, et al.
Published: (2025)

Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses
by: Rupprecht, Jens, et al.
Published: (2025)

Extracting Affect Aggregates from Longitudinal Social Media Data with Temporal Adapters for Large Language Models
by: Ahnert, Georg, et al.
Published: (2024)

Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models
by: Ahnert, Georg, et al.
Published: (2025)

Evaluating Large Language Models with Psychometrics
by: Li, Yuan, et al.
Published: (2024)

Demographic Biases and Gaps in the Perception of Sexism in Large Language Models
by: Tavarez-Rodríguez, Judith, et al.
Published: (2025)

QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models
by: Kreutner, Maximilian, et al.
Published: (2025)

R.U.Psycho? Robust Unified Psychometric Testing of Language Models
by: Schelb, Julian, et al.
Published: (2025)

Do Large Language Model Benchmarks Test Reliability?
by: Vendrow, Joshua, et al.
Published: (2025)

Do Large Language Models Understand Morality Across Cultures?
by: Mohammadi, Hadi, et al.
Published: (2025)

Do Emotions Influence Moral Judgment in Large Language Models?
by: Saim, Mohammad, et al.
Published: (2026)

Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models
by: Faulborn, Mats, et al.
Published: (2025)

Establishing Vocabulary Tests as a Benchmark for Evaluating Large Language Models
by: Martínez, Gonzalo, et al.
Published: (2023)

Smaller Large Language Models Can Do Moral Self-Correction
by: Liu, Guangliang, et al.
Published: (2024)

Toxic comments reduce the activity of volunteer editors on Wikipedia
by: Smirnov, Ivan, et al.
Published: (2023)

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks
by: Li, Peiyu, et al.
Published: (2025)

Evaluating Large Language Models with Tests of Spanish as a Foreign Language: Pass or Fail?
by: Mayor-Rocher, Marina, et al.
Published: (2024)

Do Membership Inference Attacks Work on Large Language Models?
by: Duan, Michael, et al.
Published: (2024)

Automatic Generation and Evaluation of Reading Comprehension Test Items with Large Language Models
by: Säuberli, Andreas, et al.
Published: (2024)

ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios
by: Bayer, Markus, et al.
Published: (2024)

Do Influence Functions Work on Large Language Models?
by: Li, Zhe, et al.
Published: (2024)

Evaluating Language Model Context Windows: A "Working Memory" Test and Inference-time Correction
by: Dsouza, Amanda, et al.
Published: (2024)

GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models
by: Luo, Hengyu, et al.
Published: (2025)

Large Language Models Pass the Turing Test
by: Jones, Cameron R., et al.
Published: (2025)

Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation
by: Ma, Bolei, et al.
Published: (2025)

Properties of Group Fairness Metrics for Rankings
by: Schumacher, Tobias, et al.
Published: (2022)

Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test
by: Khandelwal, Aditi, et al.
Published: (2024)

Psychometric Predictive Power of Large Language Models
by: Kuribayashi, Tatsuki, et al.
Published: (2023)

Sexism Detection on a Data Diet
by: Bandyopadhyay, Rabiraj, et al.
Published: (2024)

Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement
by: Ye, Haoran, et al.
Published: (2025)

Evaluating Gender Bias in Large Language Models
by: Döll, Michael, et al.
Published: (2024)

A Detailed Factor Analysis for the Political Compass Test: Navigating Ideologies of Large Language Models
by: Kamal, Sadia, et al.
Published: (2025)

SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
by: Vidgen, Bertie, et al.
Published: (2023)

Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness
by: Wang, Wenxuan
Published: (2024)

The Art of Scaling Test-Time Compute for Large Language Models
by: Agarwal, Aradhye, et al.
Published: (2025)