Cover Image

Who's Asking? Simulating Role-Based Questions for Conversational AI Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kaur, Navreet, Ayad, Hoda, Jung, Hayoung, Mittal, Shravika, De Choudhury, Munmun, Mitra, Tanushree
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Computers and Society Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MythTriage: Scalable Detection of Opioid Use Disorder Myths on a Video-Sharing Platform
by: Jung, Hayoung, et al.
Published: (2025)

Exposure to Content Written by Large Language Models Can Reduce Stigma Around Opioid Use Disorder in Online Communities
by: Mittal, Shravika, et al.
Published: (2025)

Algorithmic Behaviors Across Regions: A Geolocation Audit of YouTube Search for COVID-19 Misinformation Between the United States and South Africa
by: Jung, Hayoung, et al.
Published: (2024)

Employing Social Media to Improve Mental Health Outcomes
by: De Choudhury, Munmun
Published: (2025)

"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations
by: Dammu, Preetam Prabhu Srikar, et al.
Published: (2024)

The Impact and Opportunities of Generative AI in Fact-Checking
by: Wolfe, Robert, et al.
Published: (2024)

ABLEIST: Intersectional Disability Bias in LLM-Generated Hiring Scenarios
by: Phutane, Mahika, et al.
Published: (2025)

AI as We Describe It: How Large Language Models and Their Applications in Health are Represented Across Channels of Public Discourse
by: Zhou, Jiawei, et al.
Published: (2025)

The Role of Partisan Culture in Mental Health Language Online
by: Pendse, Sachin R., et al.
Published: (2025)

Evaluating Large Language Models for Health-related Queries with Presuppositions
by: Kaur, Navreet, et al.
Published: (2023)

Towards Experience-Centered AI: A Framework for Integrating Lived Experience in Design and Development
by: Gautam, Sanjana, et al.
Published: (2025)

A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models
by: Verma, Gaurav, et al.
Published: (2025)

From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents
by: Chandra, Mohit, et al.
Published: (2024)

The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support
by: Song, Inhwa, et al.
Published: (2024)

Who Does Your AI Work For? Designing Conversational Agents as Digital Fiduciaries
by: Erickson, Jacob
Published: (2026)

The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations
by: Wolfe, Robert, et al.
Published: (2024)

Synthetic Patients: Simulating Difficult Conversations with Multimodal Generative AI for Medical Education
by: Chu, Simon N., et al.
Published: (2024)

Asking For It: Question-Answering for Predicting Rule Infractions in Online Content Moderation
by: Samory, Mattia, et al.
Published: (2025)

Who Defines "Best"? Towards Interactive, User-Defined Evaluation of LLM Leaderboards
by: Jung, Minji, et al.
Published: (2026)

The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
by: Kuehnert, Blaine, et al.
Published: (2025)

Sakshm AI: Advancing AI-Assisted Coding Education for Engineering Students in India Through Socratic Tutoring and Comprehensive Feedback
by: Gupta, Raj, et al.
Published: (2025)

Who is Helping Whom? Student Concerns about AI- Teacher Collaboration in Higher Education Classrooms
by: Han, Bingyi, et al.
Published: (2024)

Who Decides How Knowing Becomes Doing? Redistributing Authority in Human-AI Music Co-Creation
by: Hu, Zhejing, et al.
Published: (2025)

PAL: Designing Conversational Agents as Scalable, Cooperative Patient Simulators for Palliative-Care Training
by: Sehgal, Neil K. R., et al.
Published: (2025)

Designing for Being-With: Presence Without Personhood in Conversational Human-AI Interaction
by: Fried, Hector Michael, et al.
Published: (2026)

Bowling with ChatGPT: On the Evolving User Interactions with Conversational AI Systems
by: Karnam, Sai Keerthana, et al.
Published: (2026)

User Privacy Harms and Risks in Conversational AI: A Proposed Framework
by: Gumusel, Ece, et al.
Published: (2024)

Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policing
by: Haque, MD Romael, et al.
Published: (2024)

Lessons from Skill Development Programs -- Livelihood College of Dhamtari
by: Choudhury, Arnab Paul, et al.
Published: (2026)

"Till I can get my satisfaction": Open Questions in the Public Desire to Punish AI
by: Ungless, Eddie L., et al.
Published: (2025)

Follow the Rules (or Not): Community Norms and AI-Generated Support in Online Health Communities
by: Mittal, Shravika, et al.
Published: (2026)

The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
by: Ehsan, Upol, et al.
Published: (2021)

Opting Out of Generative AI: a Behavioral Experiment on the Role of Education in Perplexity AI Avoidance
by: Ulloa, Roberto, et al.
Published: (2025)

Measuring Perceptions of Fairness in AI Systems: The Effects of Infra-marginality
by: Tong, Schrasing, et al.
Published: (2026)

EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders
by: Luo, Lin, et al.
Published: (2024)

Provocation: Who benefits from "inclusion" in Generative AI?
by: Dalal, Samantha, et al.
Published: (2024)

Insights from Farmer-Managed Decentralized Solar Irrigation Systems
by: Choudhury, Arnab Paul, et al.
Published: (2026)

Experiencer, Helper, or Observer: Online Fraud Intervention for Older Adults Through Role-based Simulation
by: Deng, Yue, et al.
Published: (2026)

Conversations with AI Chatbots Increase Short-Term Vaccine Intentions But Do Not Outperform Standard Public Health Messaging
by: Sehgal, Neil K. R., et al.
Published: (2025)

Student Development Agent: Risk-free Simulation for Evaluating AIED Innovations
by: Jiang, Jianxiao, et al.
Published: (2025)