A Confidence-based Acquisition Model for Self-supervised Active Learning and Label Correction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910862925627392 |
|---|---|
| author | van Niekerk, Carel Geishauser, Christian Heck, Michael Feng, Shutong Lin, Hsien-chin Lubis, Nurul Ruppik, Benjamin Vukovic, Renato Gašić, Milica |
| author_facet | van Niekerk, Carel Geishauser, Christian Heck, Michael Feng, Shutong Lin, Hsien-chin Lubis, Nurul Ruppik, Benjamin Vukovic, Renato Gašić, Milica |
| contents | Supervised neural approaches are hindered by their dependence on large, meticulously annotated datasets, a requirement that is particularly cumbersome for sequential tasks. The quality of annotations tends to deteriorate with the transition from expert-based to crowd-sourced labelling. To address these challenges, we present CAMEL (Confidence-based Acquisition Model for Efficient self-supervised active Learning), a pool-based active learning framework tailored to sequential multi-output problems. CAMEL possesses two core features: (1) it requires expert annotators to label only a fraction of a chosen sequence, and (2) it facilitates self-supervision for the remainder of the sequence. By deploying a label correction mechanism, CAMEL can also be utilised for data cleaning. We evaluate CAMEL on two sequential tasks, with a special emphasis on dialogue belief tracking, a task plagued by the constraints of limited and noisy datasets. Our experiments demonstrate that CAMEL significantly outperforms the baselines in terms of efficiency. Furthermore, the data corrections suggested by our method contribute to an overall improvement in the quality of the resulting datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_08944 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Confidence-based Acquisition Model for Self-supervised Active Learning and Label Correction van Niekerk, Carel Geishauser, Christian Heck, Michael Feng, Shutong Lin, Hsien-chin Lubis, Nurul Ruppik, Benjamin Vukovic, Renato Gašić, Milica Computation and Language Machine Learning Supervised neural approaches are hindered by their dependence on large, meticulously annotated datasets, a requirement that is particularly cumbersome for sequential tasks. The quality of annotations tends to deteriorate with the transition from expert-based to crowd-sourced labelling. To address these challenges, we present CAMEL (Confidence-based Acquisition Model for Efficient self-supervised active Learning), a pool-based active learning framework tailored to sequential multi-output problems. CAMEL possesses two core features: (1) it requires expert annotators to label only a fraction of a chosen sequence, and (2) it facilitates self-supervision for the remainder of the sequence. By deploying a label correction mechanism, CAMEL can also be utilised for data cleaning. We evaluate CAMEL on two sequential tasks, with a special emphasis on dialogue belief tracking, a task plagued by the constraints of limited and noisy datasets. Our experiments demonstrate that CAMEL significantly outperforms the baselines in terms of efficiency. Furthermore, the data corrections suggested by our method contribute to an overall improvement in the quality of the resulting datasets. |
| title | A Confidence-based Acquisition Model for Self-supervised Active Learning and Label Correction |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2310.08944 |