Copertina

Annotator in the Loop: A Case Study of In-Depth Rater Engagement to Create a Bridging Benchmark Dataset

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Schmer-Galunder, Sonja, Wheelock, Ruta, Friedman, Scott, Chvasta, Alyssa, Jalan, Zaria, Saltz, Emily
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computers and Society
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Re-Ranking News Comments by Constructiveness and Curiosity Significantly Increases Perceived Respect, Trustworthiness, and Interest
di: Saltz, Emily, et al.
Pubblicazione: (2024)

Directed Social Regard: Surfacing Targeted Advocacy, Opposition, Aid, Harms, and Victimization in Online Media
di: Friedman, Scott, et al.
Pubblicazione: (2026)

Relating Word Embedding Gender Biases to Gender Gaps: A Cross-Cultural Analysis
di: Friedman, Scott, et al.
Pubblicazione: (2026)

Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Healthcare Professionals
di: Choudhury, Avishek, et al.
Pubblicazione: (2024)

Bottom-Up and Top-Down Analysis of Values, Agendas, and Observations in Corpora and LLMs
di: Friedman, Scott E., et al.
Pubblicazione: (2024)

AI and Suicide Prevention: A Cross-Sector Primer
di: Saltz, Emily, et al.
Pubblicazione: (2026)

Discipline and Label: A WEIRD Genealogy and Social Theory of Data Annotation
di: Smart, Andrew, et al.
Pubblicazione: (2024)

SLIDE: Sliding Localized Information for Document Extraction
di: Singh, Divyansh, et al.
Pubblicazione: (2025)

Community Notes are Vulnerable to Rater Bias and Manipulation
di: Truong, Bao Tran, et al.
Pubblicazione: (2025)

Balancing Transparency and Accuracy: A Comparative Analysis of Rule-Based and Deep Learning Models in Political Bias Classification
di: Martinez, Manuel Nunez, et al.
Pubblicazione: (2024)

AI-in-the-Loop Planning for Transportation Electrification: Case Studies from Austin, Texas
di: Choi, Seung Jun
Pubblicazione: (2025)

Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks
di: Schroeder, Hope, et al.
Pubblicazione: (2025)

Applications of Artificial Intelligence Tools to Enhance Legislative Engagement: Case Studies from Make.Org and MAPLE
di: Combaz, Alicia, et al.
Pubblicazione: (2025)

Investigating the Influence of Election Debates on Children’s Political Engagement: A Case Study of the Dutch Parliamentary Elections
di: Lott H. S. Fransen, et al.
Pubblicazione: (2026)

AI and the Future of Digital Public Squares
di: Goldberg, Beth, et al.
Pubblicazione: (2024)

A Bridge to Nowhere: A Healthcare Case Study for Non-Reformist Design
di: Huber, Linda
Pubblicazione: (2025)

Predicting Healthcare Provider Engagement in SMS Campaigns
di: Qureshi, Daanish Aleem, et al.
Pubblicazione: (2025)

A High School Camp on Algorithms and Coding in Jamaica
di: Fokum, Daniel T., et al.
Pubblicazione: (2023)

WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation Detection
di: Santos, Jônatas H. dos, et al.
Pubblicazione: (2026)

Decoding Safety Feedback from Diverse Raters: A Data-driven Lens on Responsiveness to Severity
di: Mishra, Pushkar, et al.
Pubblicazione: (2025)

How to Drill Into Silos: Creating a Free-to-Use Dataset of Data Subject Access Packages
di: Leschke, Nicola, et al.
Pubblicazione: (2024)

Is GPT-4 Alone Sufficient for Automated Essay Scoring?: A Comparative Judgment Approach Based on Rater Cognition
di: Kim, Seungju, et al.
Pubblicazione: (2024)

Social Bias in Popular Question-Answering Benchmarks
di: Kraft, Angelie, et al.
Pubblicazione: (2025)

Crowdsourcing Dermatology Images with Google Search Ads: Creating a Real-World Skin Condition Dataset
di: Ward, Abbi, et al.
Pubblicazione: (2024)

Predicting Disagreement with Human Raters in LLM-as-a-Judge Difficulty Assessment without Using Generation-Time Probability Signals
di: Ehara, Yo
Pubblicazione: (2026)

Datasets for Fairness in Language Models: An In-Depth Survey
di: Zhang, Jiale, et al.
Pubblicazione: (2025)

Teaching AI Interactively: A Case Study in Higher Education
di: Reddig, Jennifer M., et al.
Pubblicazione: (2026)

Demographic Benchmarking: Bridging Socio-Technical Gaps in Bias Detection
di: Clavell, Gemma Galdon, et al.
Pubblicazione: (2025)

Comparing impacts of corn residue removal and amelioration practices on soil properties after 3, 6, and 10 years
di: Hans W. Klopp, et al.
Pubblicazione: (2024)

Annotating the Chain-of-Thought: A Behavior-Labeled Dataset for AI Safety
di: Menke, Antonio-Gabriel Chacón, et al.
Pubblicazione: (2025)

GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction
di: Felkner, Virginia K., et al.
Pubblicazione: (2024)

The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track
di: Bhardwaj, Eshta, et al.
Pubblicazione: (2024)

The Memecoin Phenomenon: An In-Depth Study of Solana's Blockchain Trends
di: Mancino, Davide
Pubblicazione: (2025)

SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations
di: Berriche, Manon, et al.
Pubblicazione: (2025)

Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
di: Denton, Remi, et al.
Pubblicazione: (2020)

BaBE: Enhancing Fairness via Estimation of Latent Explaining Variables
di: Binkyte, Ruta, et al.
Pubblicazione: (2023)

On the Need and Applicability of Causality for Fairness: A Unified Framework for AI Auditing and Legal Analysis
di: Binkyte, Ruta, et al.
Pubblicazione: (2022)

Influence of Water Vapor Content on Oxidation of the SiC Layer of Tristructural Isotropic Particles
di: Visharad Jalan, et al.
Pubblicazione: (2026)

Social Engagement and Epigenetic Age Acceleration in the Health and Retirement Study
di: Zhao Hu, et al.
Pubblicazione: (2026)

Instructor-Created Custom GPTs as Pedagogical Partners Fostering Immersion in Online Higher Education: Two Case Studies
di: Beck, Dennis, et al.
Pubblicazione: (2026)