Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.04637 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910800752410624 |
|---|---|
| author | Artemova, Ekaterina Tsvigun, Akim Schlechtweg, Dominik Fedorova, Natalia Chernyshev, Konstantin Tilga, Sergei Obmoroshev, Boris |
| author_facet | Artemova, Ekaterina Tsvigun, Akim Schlechtweg, Dominik Fedorova, Natalia Chernyshev, Konstantin Tilga, Sergei Obmoroshev, Boris |
| contents | Training and deploying machine learning models relies on a large amount of human-annotated data. As human labeling becomes increasingly expensive and time-consuming, recent research has developed multiple strategies to speed up annotation and reduce costs and human workload: generating synthetic training data, active learning, and hybrid labeling. This tutorial is oriented toward practical applications: we will present the basics of each strategy, highlight their benefits and limitations, and discuss in detail real-life case studies. Additionally, we will walk through best practices for managing human annotators and controlling the quality of the final dataset. The tutorial includes a hands-on workshop, where attendees will be guided in implementing a hybrid annotation setup. This tutorial is designed for NLP practitioners from both research and industry backgrounds who are involved in or interested in optimizing data labeling projects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04637 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hands-On Tutorial: Labeling with LLM and Human-in-the-Loop Artemova, Ekaterina Tsvigun, Akim Schlechtweg, Dominik Fedorova, Natalia Chernyshev, Konstantin Tilga, Sergei Obmoroshev, Boris Computation and Language Training and deploying machine learning models relies on a large amount of human-annotated data. As human labeling becomes increasingly expensive and time-consuming, recent research has developed multiple strategies to speed up annotation and reduce costs and human workload: generating synthetic training data, active learning, and hybrid labeling. This tutorial is oriented toward practical applications: we will present the basics of each strategy, highlight their benefits and limitations, and discuss in detail real-life case studies. Additionally, we will walk through best practices for managing human annotators and controlling the quality of the final dataset. The tutorial includes a hands-on workshop, where attendees will be guided in implementing a hybrid annotation setup. This tutorial is designed for NLP practitioners from both research and industry backgrounds who are involved in or interested in optimizing data labeling projects. |
| title | Hands-On Tutorial: Labeling with LLM and Human-in-the-Loop |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.04637 |