Copertina

A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Li, Jiyi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Human-Computer Interaction
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Human-LLM Hybrid Text Answer Aggregation for Crowd Annotations
di: Li, Jiyi
Pubblicazione: (2024)

MEGAnno+: A Human-LLM Collaborative Annotation System
di: Kim, Hannah, et al.
Pubblicazione: (2024)

Evaluating Saliency Explanations in NLP by Crowdsourcing
di: Lu, Xiaotian, et al.
Pubblicazione: (2024)

Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems
di: Siro, Clemencia, et al.
Pubblicazione: (2024)

Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twins
di: Chan, Amanda, et al.
Pubblicazione: (2025)

Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflows
di: Grunde-McLaughlin, Madeleine, et al.
Pubblicazione: (2023)

If in a Crowdsourced Data Annotation Pipeline, a GPT-4
di: He, Zeyu, et al.
Pubblicazione: (2024)

Efficiency and Effectiveness of LLM-Based Summarization of Evidence in Crowdsourced Fact-Checking
di: Roitero, Kevin, et al.
Pubblicazione: (2025)

LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs
di: Wu, Tongshuang, et al.
Pubblicazione: (2023)

Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learning
di: Qiu, Wenjun, et al.
Pubblicazione: (2024)

Efficiently Crowdsourcing Visual Importance with Punch-Hole Annotation
di: Chang, Minsuk, et al.
Pubblicazione: (2024)

Crowdsourced Adaptive Surveys
di: Velez, Yamil
Pubblicazione: (2024)

Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets
di: Giorgi, Tommaso, et al.
Pubblicazione: (2024)

Introducing Quality Estimation to Machine Translation Post-editing Workflow: An Empirical Study on Its Usefulness
di: Liu, Siqi, et al.
Pubblicazione: (2025)

Can Unconfident LLM Annotations Be Used for Confident Conclusions?
di: Gligorić, Kristina, et al.
Pubblicazione: (2024)

The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs
di: Calderon, Nitay, et al.
Pubblicazione: (2025)

Grid Labeling: Crowdsourcing Task-Specific Importance from Visualizations
di: Chang, Minsuk, et al.
Pubblicazione: (2025)

Efficient Online Crowdsourcing with Complex Annotations
di: Meir, Reshef, et al.
Pubblicazione: (2024)

What Makes LLM Agent Simulations Useful for Policy Practice? An Iterative Design Study in Emergency Preparedness
di: Li, Yuxuan, et al.
Pubblicazione: (2025)

EEVEE: An Easy Annotation Tool for Natural Language Processing
di: Sorensen, Axel, et al.
Pubblicazione: (2024)

ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation
di: Huang, Chen, et al.
Pubblicazione: (2024)

Personas with Attitudes: Controlling LLMs for Diverse Data Annotation
di: Fröhling, Leon, et al.
Pubblicazione: (2024)

Evaluating LLM-Generated Q&A Test: a Student-Centered Study
di: Wróblewska, Anna, et al.
Pubblicazione: (2025)

Sandpiper: Orchestrated AI-Annotation for Educational Discourse at Scale
di: Hedley, Daryl, et al.
Pubblicazione: (2026)

Navigating Rifts in Human-LLM Grounding: Study and Benchmark
di: Shaikh, Omar, et al.
Pubblicazione: (2025)

Online vs Offline: A Comparative Study of First-Party and Third-Party Evaluations of Social Chatbots
di: Svikhnushina, Ekaterina, et al.
Pubblicazione: (2024)

From Chatbots to Confidants: A Cross-Cultural Study of LLM Adoption for Emotional Support
di: Amat-Lefort, Natalia, et al.
Pubblicazione: (2026)

Improving Data Quality via Pre-Task Participant Screening in Crowdsourced GUI Experiments
di: Miyama, Takaya, et al.
Pubblicazione: (2026)

Assessing the Reliability of Large Language Models for Deductive Qualitative Coding: A Comparative Study of ChatGPT Interventions
di: Hila, Angjelin, et al.
Pubblicazione: (2025)

Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP
di: Gautam, Sanjana, et al.
Pubblicazione: (2024)

Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement
di: Sarti, Gabriele, et al.
Pubblicazione: (2025)

RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systems
di: Tian, Haoyu, et al.
Pubblicazione: (2026)

MapQaTor: An Extensible Framework for Efficient Annotation of Map-Based QA Datasets
di: Dihan, Mahir Labib, et al.
Pubblicazione: (2024)

Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant
di: He, Gaole, et al.
Pubblicazione: (2025)

Visualization Literacy of Multimodal Large Language Models: A Comparative Study
di: Li, Zhimin, et al.
Pubblicazione: (2024)

A Survey on LLM-based Conversational User Simulation
di: Ni, Bo, et al.
Pubblicazione: (2026)

UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents
di: Lu, Yuxuan, et al.
Pubblicazione: (2025)

Simulating Classroom Education with LLM-Empowered Agents
di: Zhang, Zheyuan, et al.
Pubblicazione: (2024)

Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles
di: Louie, Ryan, et al.
Pubblicazione: (2024)

A-MEM: Agentic Memory for LLM Agents
di: Xu, Wujiang, et al.
Pubblicazione: (2025)