Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue History

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhong, Qishuai, Li, Zongmin, Fan, Siqi, Sun, Aixin
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

An Analysis of User Behaviors for Objectively Evaluating Spoken Dialogue Systems
by: Inoue, Koji, et al.
Published: (2024)

User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
by: Wu, Xiaoyuan, et al.
Published: (2025)

Evaluating Behavioral Alignment in Conflict Dialogue: A Multi-Dimensional Comparison of LLM Agents and Humans
by: Kwon, Deuksin, et al.
Published: (2025)

Dialect vs Demographics: Quantifying LLM Bias from Implicit Linguistic Signals vs. Explicit User Profiles
by: Haq, Irti, et al.
Published: (2026)

Evaluating Large Language Models in Analysing Classroom Dialogue
by: Long, Yun, et al.
Published: (2024)

Collaborative Evaluation of Deepfake Text with Deliberation-Enhancing Dialogue Systems
by: Lee, Jooyoung, et al.
Published: (2025)

Understand User Opinions of Large Language Models via LLM-Powered In-the-Moment User Experience Interviews
by: Liu, Mengqiao, et al.
Published: (2025)

Estimating LLM Consistency: A User Baseline vs Surrogate Metrics
by: Wu, Xiaoyuan, et al.
Published: (2025)

PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue
by: Jeon, Hyunbae, et al.
Published: (2026)

ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
by: Liu, Tianjian, et al.
Published: (2025)

Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues
by: Cohen, Myke C., et al.
Published: (2025)

Stories of Your Life as Others: A Round-Trip Evaluation of LLM-Generated Life Stories Conditioned on Rich Psychometric Profiles
by: Wigler, Ben, et al.
Published: (2026)

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
by: Qian, Cheng, et al.
Published: (2024)

Causal Discovery and Counterfactual Reasoning to Optimize Persuasive Dialogue Policies
by: Zeng, Donghuo, et al.
Published: (2025)

Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery
by: Clark, Nicholas, et al.
Published: (2025)

User-Assistant Bias in LLMs
by: Pan, Xu, et al.
Published: (2025)

Personality as Relational Infrastructure: User Perceptions of Personality-Trait-Infused LLM Messaging
by: Hofer, Dominik P., et al.
Published: (2026)

Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences
by: Zheng, Mingqian, et al.
Published: (2025)

BADGE: BADminton report Generation and Evaluation with LLM
by: Chiang, Shang-Hsuan, et al.
Published: (2024)

EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria
by: Kim, Tae Soo, et al.
Published: (2023)

Language Models in Dialogue: Conversational Maxims for Human-AI Interactions
by: Miehling, Erik, et al.
Published: (2024)

Human-AI Interaction and User Satisfaction: Empirical Evidence from Online Reviews of AI Products
by: Pasch, Stefan, et al.
Published: (2025)

Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs
by: Sun, Huaman, et al.
Published: (2023)

HealthGenie: Empowering Users with Healthy Dietary Guidance through Knowledge Graph and Large Language Models
by: Gao, Fan, et al.
Published: (2025)

Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks
by: Rahman, Hasibur, et al.
Published: (2025)

Using Machine Mental Imagery for Representing Common Ground in Situated Dialogue
by: Mohapatra, Biswesh, et al.
Published: (2026)

Depression Diagnosis Dialogue Simulation: Self-improving Psychiatrist with Tertiary Memory
by: Lan, Kunyao, et al.
Published: (2024)

Exploring ReAct Prompting for Task-Oriented Dialogue: Insights and Shortcomings
by: Elizabeth, Michelle, et al.
Published: (2024)

Diverse and Fine-Grained Instruction-Following Ability Exploration with Synthetic Data
by: Gu, Zihui, et al.
Published: (2024)

Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets
by: Druart, Lucas, et al.
Published: (2024)

Development and Evaluation of HopeBot: an LLM-based chatbot for structured and interactive PHQ-9 depression screening
by: Guo, Zhijun, et al.
Published: (2025)

Ensembling Large Language Models to Characterize Affective Dynamics in Student-AI Tutor Dialogues
by: Zhang, Chenyu, et al.
Published: (2025)

Advancing Speech Language Models by Scaling Supervised Fine-Tuning with Over 60,000 Hours of Synthetic Speech Dialogue Data
by: Zhao, Shuaijiang, et al.
Published: (2024)

VizTrust: A Visual Analytics Tool for Capturing User Trust Dynamics in Human-AI Communication
by: Wang, Xin, et al.
Published: (2025)

Do Ethical AI Principles Matter to Users? A Large-Scale Analysis of User Sentiment and Satisfaction
by: Pasch, Stefan, et al.
Published: (2025)

VeriLA: A Human-Centered Evaluation Framework for Interpretable Verification of LLM Agent Failures
by: Sung, Yoo Yeon, et al.
Published: (2025)

Learned, Lagged, LLM-splained: LLM Responses to End User Security Questions
by: Prakash, Vijay, et al.
Published: (2024)

Keeping Users Engaged During Repeated Administration of the Same Questionnaire: Using Large Language Models to Reliably Diversify Questions
by: Yun, Hye Sun, et al.
Published: (2023)

Evaluating LLM-Generated Lessons from the Language Learning Students' Perspective: A Short Case Study on Duolingo
by: Catalan, Carlos Rafael, et al.
Published: (2026)

Creating General User Models from Computer Use
by: Shaikh, Omar, et al.
Published: (2025)