Cover Image

A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Harvey, Emma, Kizilcec, Rene F., Koenecke, Allison
Format:	Preprint
Published:	2025
Subjects:	Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

"Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Education
by: Harvey, Emma, et al.
Published: (2025)

Into the Unknown: Accounting for Missing Demographic Data when Mitigating Ad Delivery Skew
by: Corpus, Isabel, et al.
Published: (2026)

Careless Whisper: Speech-to-Text Hallucination Harms
by: Koenecke, Allison, et al.
Published: (2024)

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability
by: Lee, Jinsook, et al.
Published: (2024)

Who Decides in AI-Mediated Learning? The Agency Allocation Framework
by: Borchers, Conrad, et al.
Published: (2026)

A Critical Pragmatism Approach for Algorithmic Fairness: Lessons from Urban Planning Theory
by: Gosciak, Jennah, et al.
Published: (2026)

Teachers' Perceived Benefits and Risks of AI Across Fifty-Five Countries: An Audit of LLM Alignment and Steerability
by: Tao, Yan, et al.
Published: (2026)

Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems
by: Yan, Jing Nathan, et al.
Published: (2025)

Scrutinizing Index-Based Risk Assessments: A Case Study in NYC Decision-making for Heat Emergency Management
by: Gosciak, Jennah, et al.
Published: (2026)

Reflection on Purpose Changes Students' Academic Interests: A Scalable Intervention in an Online Course Catalog
by: Chen, Youjie, et al.
Published: (2024)

Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasia
by: Mei, Katelyn Xiaoying, et al.
Published: (2025)

Introducing AI to an Online Petition Platform Changed Outputs but not Outcomes
by: Corpus, Isabel, et al.
Published: (2025)

Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinese
by: Lyu, Hanjia, et al.
Published: (2025)

Does Algorithmic Uncertainty Sway Human Experts? Evidence from a Field Experiment in Selective College Admissions
by: Lee, Hansol, et al.
Published: (2026)

The Life Cycle of Large Language Models: A Review of Biases in Education
by: Lee, Jinsook, et al.
Published: (2024)

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
by: Harvey, Emma, et al.
Published: (2024)

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
by: Harvey, Emma, et al.
Published: (2025)

Harmful Speech Detection by Language Models Exhibits Gender-Queer Dialect Bias
by: Dorn, Rebecca, et al.
Published: (2024)

LLMs in social services: How does chatbot accuracy affect human accuracy?
by: Gosciak, Jennah, et al.
Published: (2026)

From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguards
by: Chehbouni, Khaoula, et al.
Published: (2024)

When the Past Misleads: Rethinking Training Data Expansion Under Temporal Distribution Shifts
by: Yao, Chengyuan, et al.
Published: (2025)

The Digital Divide in Generative AI: Evidence from Large Language Model Use in College Admissions Essays
by: Lee, Jinsook, et al.
Published: (2026)

Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments
by: Gosciak, Jennah, et al.
Published: (2025)

Investigating Student Interaction Patterns with Large Language Model-Powered Course Assistants in Computer Science Courses
by: Liu, Chang, et al.
Published: (2025)

Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education
by: Thomas, Danielle R., et al.
Published: (2026)

Fairness Hub Technical Briefs: Definition and Detection of Distribution Shift
by: Acevedo, Nicolas, et al.
Published: (2024)

Beyond Performance Disparities: A Three-Level Audit of Representational Harm in CelebA
by: Park, Sieun, et al.
Published: (2026)

Harm in AI-Driven Societies: An Audit of Toxicity Adoption on Chirper.ai
by: Coppolillo, Erica, et al.
Published: (2026)

A Framework for Assurance Audits of Algorithmic Systems
by: Lam, Khoa, et al.
Published: (2024)

Cultural Differences in Students' Privacy Concerns in Learning Analytics across Germany, South Korea, Spain, Sweden, and the United States
by: Viberg, Olga, et al.
Published: (2023)

Gaming the Metric, Not the Harm: Certifying Safety Audits against Strategic Platform Manipulation
by: Burnat, Florian A. D., et al.
Published: (2026)

Echoes of AI Harms: A Human-LLM Synergistic Framework for Bias-Driven Harm Anticipation
by: Tantalaki, Nicoleta, et al.
Published: (2025)

A Canary in the AI Coal Mine: American Jews May Be Disproportionately Harmed by Intellectual Property Dispossession in Large Language Model Training
by: Precel, Heila, et al.
Published: (2024)

Analyzing Dialectical Biases in LLMs for Knowledge and Reasoning Benchmarks
by: Pan, Eileen, et al.
Published: (2025)

Potential for allocative harm in an environmental justice data tool
by: Huynh, Benjamin Q., et al.
Published: (2023)

Comprehensive Framework for Evaluating Conversational AI Chatbots
by: Gupta, Shailja, et al.
Published: (2025)

Doing Audits Right? The Role of Sampling and Legal Content Analysis in Systemic Risk Assessments and Independent Audits in the Digital Services Act
by: Sekwenz, Marie-Therese, et al.
Published: (2025)

From Individual Experience to Collective Evidence: A Reporting-Based Framework for Identifying Systemic Harms
by: Dai, Jessica, et al.
Published: (2025)

A Mixed-Methods Evaluation of LLM-Based Chatbots for Menopause
by: Deva, Roshini, et al.
Published: (2025)

User Privacy Harms and Risks in Conversational AI: A Proposed Framework
by: Gumusel, Ece, et al.
Published: (2024)