Imagen de Portada

Principled Evaluation with Human Labels: One Rater at a Time and Rater Equivalence

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Resnick, Paul, Kong, Yuqing, Schoenebeck, Grant, Weninger, Tim
Formato:	Preprint
Publicado:	2021
Materias:	Machine Learning Human-Computer Interaction Multiagent Systems
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Human-in-the-Loop Testing of AI Agents for Air Traffic Control with a Regulated Assessment Framework
por: Carvell, Ben, et al.
Publicado: (2026)

Enabling Multi-Robot Collaboration from Single-Human Guidance
por: Ji, Zhengran, et al.
Publicado: (2024)

HRI-SA: A Multimodal Dataset for Online Assessment of Human Situational Awareness during Remote Human-Robot Teaming
por: Senaratne, Hashini, et al.
Publicado: (2026)

Multi-Turn Human-LLM Interaction Through the Lens of a Two-Way Intelligibility Protocol
por: Mestha, Harshvardhan, et al.
Publicado: (2024)

Conformal Set-based Human-AI Complementarity with Multiple Experts
por: Paat, Helbert, et al.
Publicado: (2025)

Socratic: Enhancing Human Teamwork via AI-enabled Coaching
por: Seo, Sangwon, et al.
Publicado: (2025)

Agent-Based Modular Learning for Multimodal Emotion Recognition in Human-Agent Systems
por: Nepomnyaschiy, Matvey, et al.
Publicado: (2025)

Fully Data-driven but Interpretable Human Behavioural Modelling with Differentiable Discrete Choice Model
por: Makinoshima, Fumiyasu, et al.
Publicado: (2024)

HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learning
por: Hu, Huawen, et al.
Publicado: (2024)

ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination
por: Wang, Xihuai, et al.
Publicado: (2023)

A Future Capabilities Agent for Tactical Air Traffic Control
por: Kent, Paul, et al.
Publicado: (2026)

Towards Human Engagement with Realistic AI Combat Pilots
por: Selmonaj, Ardian, et al.
Publicado: (2025)

MAS-KCL: Knowledge component graph structure learning with large language model-based agentic workflow
por: Jiang, Yuan-Hao, et al.
Publicado: (2025)

Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot Teaming
por: Angelotti, Giorgio, et al.
Publicado: (2024)

Analyzing Operator States and the Impact of AI-Enhanced Decision Support in Control Rooms: A Human-in-the-Loop Specialized Reinforcement Learning Framework for Intervention Strategies
por: Abbas, Ammar N., et al.
Publicado: (2024)

xSRL: Safety-Aware Explainable Reinforcement Learning -- Safety as a Product of Explainability
por: Shefin, Risal Shahriar, et al.
Publicado: (2024)

Combating Spatial Disorientation in a Dynamic Self-Stabilization Task Using AI Assistants
por: Mannan, Sheikh, et al.
Publicado: (2024)

The Patterns of Life Human Mobility Simulation
por: Amiri, Hossein, et al.
Publicado: (2024)

A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomy
por: Zou, Henry Peng, et al.
Publicado: (2025)

Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration
por: Zhang, Shao, et al.
Publicado: (2025)

MemArchitect: A Policy Driven Memory Governance Layer
por: Kumar, Lingavasan Suresh, et al.
Publicado: (2026)

Bidirectional human-AI collaboration in brain tumour assessments improves both expert human and AI agent performance
por: Ruffle, James K, et al.
Publicado: (2025)

From Governance Norms to Enforceable Controls: A Layered Translation Method for Runtime Guardrails in Agentic AI
por: Koch, Christopher
Publicado: (2026)

KOM: A Multi-Agent Artificial Intelligence System for Precision Management of Knee Osteoarthritis (KOA)
por: Liu, Weizhi, et al.
Publicado: (2025)

Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
por: Ruhdorfer, Constantin, et al.
Publicado: (2025)

Plural Voices, Single Agent: Towards Inclusive AI in Multi-User Domestic Spaces
por: Chandra, Joydeep, et al.
Publicado: (2025)

A2H: Agent-to-Human Protocol for AI Agent
por: Liang, Zhiyuan, et al.
Publicado: (2025)

On the Utility of External Agent Intention Predictor for Human-AI Coordination
por: Wang, Chenxu, et al.
Publicado: (2024)

ALICE: Combining Feature Selection and Inter-Rater Agreeability for Machine Learning Insights
por: Anasashvili, Bachana, et al.
Publicado: (2024)

How to Steer Your Multi-Agent System: Human-LLM Collaborative Planning
por: He, Zeyu, et al.
Publicado: (2026)

Cloud and IoT based Smart Agent-driven Simulation of Human Gait for Detecting Muscles Disorder
por: Saadati, Sina, et al.
Publicado: (2024)

FlowEval: Reference-based Evaluation of Generated User Interfaces
por: Wu, Jason, et al.
Publicado: (2026)

CandorMD: An AI-Assisted Audio Simulation and Feedback System for Training Clinicians for Medical Error Disclosure
por: Lin, Inna Wanyin, et al.
Publicado: (2026)

IronEngine: Towards General AI Assistant
por: Mo, Xi
Publicado: (2026)

Conjugated Capabilities: Interrelations of Elementary Human Capabilities and Their Implication on Human-Machine Task Allocation and Capability Testing Procedures
por: Mandischer, Nils, et al.
Publicado: (2025)

Asynchronous Training of Mixed-Role Human Actors in a Partially-Observable Environment
por: Chang, Kimberlee Chestnut, et al.
Publicado: (2024)

GPT-4's assessment of its performance in a USMLE-based case study
por: Dhakal, Uttam, et al.
Publicado: (2024)

Speculative Ad-hoc Querying
por: Li, Haoyu, et al.
Publicado: (2025)

Coordinated Strategies in Realistic Air Combat by Hierarchical Multi-Agent Reinforcement Learning
por: Selmonaj, Ardian, et al.
Publicado: (2025)

PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents
por: Yang, Qisen, et al.
Publicado: (2024)