Cover Image

AI vs. Human Judgment of Content Moderation: LLM-as-a-Judge and Ethics-Based Response Refusals

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Pasch, Stefan
Format:	Preprint
Published:	2025
Subjects:	Human-Computer Interaction Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arena
by: Pasch, Stefan
Published: (2025)

Human-AI Interaction and User Satisfaction: Empirical Evidence from Online Reviews of AI Products
by: Pasch, Stefan, et al.
Published: (2025)

Bottom-Up Perspectives on AI Governance: Insights from User Reviews of AI Products
by: Pasch, Stefan
Published: (2025)

Do Ethical AI Principles Matter to Users? A Large-Scale Analysis of User Sentiment and Satisfaction
by: Pasch, Stefan, et al.
Published: (2025)

Operationalizing AI: Empirical Evidence on MLOps Practices, User Satisfaction, and Organizational Context
by: Pasch, Stefan
Published: (2025)

User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
by: Wu, Xiaoyuan, et al.
Published: (2025)

Longitudinal Monitoring of LLM Content Moderation of Social Issues
by: Dai, Yunlang, et al.
Published: (2025)

Identity-related Speech Suppression in Generative AI Content Moderation
by: Proebsting, Grace, et al.
Published: (2024)

Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated
by: Zhu, Tiffany, et al.
Published: (2024)

Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations
by: Hartmann, David, et al.
Published: (2025)

Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale
by: Li, Weiyue, et al.
Published: (2026)

Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild
by: Mysore, Sheshera, et al.
Published: (2025)

The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs
by: Calderon, Nitay, et al.
Published: (2025)

AI Content Moderation in Therapy Conversations
by: Kim, Jiwon, et al.
Published: (2026)

Creative Beam Search: LLM-as-a-Judge For Improving Response Generation
by: Franceschelli, Giorgio, et al.
Published: (2024)

The Hidden Language of Harm: Examining the Role of Emojis in Harmful Online Communication and Content Moderation
by: Zhou, Yuhang, et al.
Published: (2025)

Sniff AI: Is My 'Spicy' Your 'Spicy'? Exploring LLM's Perceptual Alignment with Human Smell Experiences
by: Zhong, Shu, et al.
Published: (2024)

Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scale
by: Lang, Max M., et al.
Published: (2025)

Human-LLM Collaborative Construction of a Cantonese Emotion Lexicon
by: Zhang, Yusong, et al.
Published: (2024)

Between Rules and Reality: On the Context Sensitivity of LLM Moral Judgment
by: Sauter, Adrian, et al.
Published: (2026)

Automated Bias Assessment in AI-Generated Educational Content Using CEAT Framework
by: Peng, Jingyang, et al.
Published: (2025)

Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?
by: Kim, Jane Paik
Published: (2026)

Navigating Rifts in Human-LLM Grounding: Study and Benchmark
by: Shaikh, Omar, et al.
Published: (2025)

MEGAnno+: A Human-LLM Collaborative Annotation System
by: Kim, Hannah, et al.
Published: (2024)

LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?
by: Sun, Lu, et al.
Published: (2025)

Product vs. Process: Exploring EFL Students' Editing of AI-Generated Text for Expository Writing
by: Woo, David James, et al.
Published: (2025)

EmoXpt: Analyzing Emotional Variances in Human Comments and LLM-Generated Responses
by: Pyreddy, Shireesh Reddy, et al.
Published: (2025)

Lexical Indicators of Mind Perception in Human-AI Companionship
by: Banks, Jaime, et al.
Published: (2026)

When Large Language Models are Reliable for Judging Empathic Communication
by: Kumar, Aakriti, et al.
Published: (2025)

Human-Centered Design Recommendations for LLM-as-a-Judge
by: Pan, Qian, et al.
Published: (2024)

Interview AI-ssistant: Designing for Real-Time Human-AI Collaboration in Interview Preparation and Execution
by: Liu, Zhe
Published: (2025)

Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models
by: Duan, Ranjie, et al.
Published: (2025)

Show or Tell? Modeling the evolution of request-making in Human-LLM conversations
by: Zhu, Shengqi, et al.
Published: (2025)

Overreliance on AI in Information-seeking from Video Content
by: Møller, Anders Giovanni, et al.
Published: (2026)

Althea: Human-AI Collaboration for Fact-Checking and Critical Reasoning
by: Churina, Svetlana, et al.
Published: (2025)

Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
by: Zhang, Xingjian, et al.
Published: (2025)

Who is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition
by: Xi, Sarina, et al.
Published: (2025)

Beyond Turn-taking: Introducing Text-based Overlap into Human-LLM Interactions
by: Kim, JiWoo, et al.
Published: (2025)

The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?
by: Choi, Alexander S., et al.
Published: (2024)

Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts
by: Li, Yaqiong, et al.
Published: (2026)