ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917681652826112 |
|---|---|
| author | Huang, Chen Jin, Yiping Ilievski, Ilija Lei, Wenqiang Lv, Jiancheng |
| author_facet | Huang, Chen Jin, Yiping Ilievski, Ilija Lei, Wenqiang Lv, Jiancheng |
| contents | Human annotation is a time-consuming task that requires a significant amount of effort. To address this issue, interactive data annotation utilizes an annotation model to provide suggestions for humans to approve or correct. However, annotation models trained with limited labeled data are prone to generating incorrect suggestions, leading to extra human correction effort. To tackle this challenge, we propose Araida, an analogical reasoning-based approach that enhances automatic annotation accuracy in the interactive data annotation setting and reduces the need for human corrections. Araida involves an error-aware integration strategy that dynamically coordinates an annotation model and a k-nearest neighbors (KNN) model, giving more importance to KNN's predictions when predictions from the annotation model are deemed inaccurate. Empirical studies demonstrate that Araida is adaptable to different annotation tasks and models. On average, it reduces human correction labor by 11.02% compared to vanilla interactive data annotation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11912 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation Huang, Chen Jin, Yiping Ilievski, Ilija Lei, Wenqiang Lv, Jiancheng Computation and Language Human-Computer Interaction Human annotation is a time-consuming task that requires a significant amount of effort. To address this issue, interactive data annotation utilizes an annotation model to provide suggestions for humans to approve or correct. However, annotation models trained with limited labeled data are prone to generating incorrect suggestions, leading to extra human correction effort. To tackle this challenge, we propose Araida, an analogical reasoning-based approach that enhances automatic annotation accuracy in the interactive data annotation setting and reduces the need for human corrections. Araida involves an error-aware integration strategy that dynamically coordinates an annotation model and a k-nearest neighbors (KNN) model, giving more importance to KNN's predictions when predictions from the annotation model are deemed inaccurate. Empirical studies demonstrate that Araida is adaptable to different annotation tasks and models. On average, it reduces human correction labor by 11.02% compared to vanilla interactive data annotation methods. |
| title | ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation |
| topic | Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2405.11912 |