Image de couverture de livre

MultEval: Supporting Collaborative Alignment for LLM-as-a-Judge Evaluation Criteria

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chiang, Charles, Gebreegziabher, Simret, Szymanski, Annalisa, Yang, Yukun, Do, Hyo Jin, Ashktorab, Zahra, Geyer, Werner, Li, Toby, Gomez-Zara, Diego
Format:	Preprint
Publié:	2026
Sujets:	Human-Computer Interaction
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes
par: Gebreegziabher, Simret Araya, et autres
Publié: (2026)

Designing Staged Evaluation Workflows for LLMs: Integrating Domain Experts, Lay Users, and Model-Generated Evaluation Criteria
par: Szymanski, Annalisa, et autres
Publié: (2024)

EvalAssist: A Human-Centered Tool for LLM-as-a-Judge
par: Ashktorab, Zahra, et autres
Publié: (2025)

Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories
par: Gebreegziabher, Simret Araya, et autres
Publié: (2024)

Stayin' Aligned Over Time: Towards Longitudinal Human-LLM Alignment via Contextual Reflection and Privacy-Preserving Behavioral Data
par: Gebreegziabher, Simret Araya, et autres
Publié: (2026)

Generate, Evaluate, Iterate: Synthetic Data for Human-in-the-Loop Refinement of LLM Judges
par: Do, Hyo Jin, et autres
Publié: (2025)

Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences
par: Ashktorab, Zahra, et autres
Publié: (2024)

Human-Centered Design Recommendations for LLM-as-a-Judge
par: Pan, Qian, et autres
Publié: (2024)

Hide or Highlight: Understanding the Impact of Factuality Expression on User Trust
par: Do, Hyo Jin, et autres
Publié: (2025)

Black-box Uncertainty Quantification Method for LLM-as-a-Judge
par: Wagner, Nico, et autres
Publié: (2024)

Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning
par: Gebreegziabher, Simret Araya, et autres
Publié: (2024)

Helping the Helper: Supporting Peer Counselors via AI-Empowered Practice and Feedback
par: Hsu, Shang-Ling, et autres
Publié: (2023)

Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks
par: Szymanski, Annalisa, et autres
Publié: (2024)

A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration
par: Gao, Jie, et autres
Publié: (2024)

The Evolution of Emojis for Sharing Emotions: A Systematic Review of the HCI Literature
par: Chiang, Charles, et autres
Publié: (2024)

Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions
par: Do, Hyo Jin, et autres
Publié: (2024)

Interaction Configurations and Prompt Guidance in Conversational AI for Question Answering in Human-AI Teams
par: Song, Jaeyoon, et autres
Publié: (2025)

CollabEval: Enhancing LLM-as-a-Judge via Multi-Agent Collaboration
par: Qian, Yiyue, et autres
Publié: (2026)

Highlight All the Phrases: Enhancing LLM Transparency through Visual Factuality Indicators
par: Do, Hyo Jin, et autres
Publié: (2025)

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study
par: Szymanski, Annalisa, et autres
Publié: (2026)

Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents
par: Chen, Chaoran, et autres
Publié: (2025)

Scopes of Alignment
par: Varshney, Kush R., et autres
Publié: (2025)

The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections
par: Chen, Chaoran, et autres
Publié: (2025)

Emerging Reliance Behaviors in Human-AI Content Grounded Data Generation: The Role of Cognitive Forcing Functions and Hallucinations
par: Ashktorab, Zahra, et autres
Publié: (2024)

From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews
par: Nogueira, Brenda, et autres
Publié: (2025)

A Case Study Investigating the Role of Generative AI in Quality Evaluations of Epics in Agile Software Development
par: Geyer, Werner, et autres
Publié: (2025)

tAIfa: Enhancing Team Effectiveness and Cohesion with AI-Generated Automated Feedback
par: Almutairi, Mohammed, et autres
Publié: (2025)

Togedule: Scheduling Meetings with Large Language Models and Adaptive Representations of Group Availability
par: Song, Jaeyoon, et autres
Publié: (2025)

EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models
par: Mohammadi, Hadi, et autres
Publié: (2025)

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight
par: Tang, Jingyu, et autres
Publié: (2025)

Augmenting Teamwork through AI Agents as Spatial Collaborators
par: Fernandez-Espinosa, Mariana, et autres
Publié: (2025)

Oral presentation of: MultKAN-Nash: Strategic Multi-Agent Disaster Response using MultKAN and Nash Equilibriums
par: Nishat Tasnin
Publié: (2025)

"Feeling that I was Collaborating with Them:" A 20-year Scoping Review of Social Virtual Reality Leveraging Collaboration
par: Sayadi, Niloofar, et autres
Publié: (2024)

CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agents
par: Wan, Ruyuan, et autres
Publié: (2024)

YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering
par: D'Souza, Jennifer, et autres
Publié: (2025)

HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
par: Yang, Langqi, et autres
Publié: (2025)

PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?
par: Zhou, Lingfeng, et autres
Publié: (2025)

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
par: Ye, Jiayi, et autres
Publié: (2024)

LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories
par: Vishnubhotla, Krishnapriya, et autres
Publié: (2026)

Augmenting team diversity and performance by enabling agency and fairness criteria in recommendation algorithms
par: Gomez-Zara, Diego, et autres
Publié: (2024)