Improving ML Training Data with Gold-Standard Quality Metrics
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918261423079424 |
|---|---|
| author | Barrett, Leslie Sherman, Michael W. |
| author_facet | Barrett, Leslie Sherman, Michael W. |
| contents | Hand-tagged training data is essential to many machine learning tasks. However, training data quality control has received little attention in the literature, despite data quality varying considerably with the tagging exercise. We propose methods to evaluate and enhance the quality of hand-tagged training data using statistical approaches to measure tagging consistency and agreement. We show that agreement metrics give more reliable results if recorded over multiple iterations of tagging, where declining variance in such recordings is an indicator of increasing data quality. We also show one way a tagging project can collect high-quality training data without requiring multiple tags for every work item, and that a tagger burn-in period may not be sufficient for minimizing tagger errors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_20577 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving ML Training Data with Gold-Standard Quality Metrics Barrett, Leslie Sherman, Michael W. Machine Learning I.2.7; I.2.6; G.3 Hand-tagged training data is essential to many machine learning tasks. However, training data quality control has received little attention in the literature, despite data quality varying considerably with the tagging exercise. We propose methods to evaluate and enhance the quality of hand-tagged training data using statistical approaches to measure tagging consistency and agreement. We show that agreement metrics give more reliable results if recorded over multiple iterations of tagging, where declining variance in such recordings is an indicator of increasing data quality. We also show one way a tagging project can collect high-quality training data without requiring multiple tags for every work item, and that a tagger burn-in period may not be sufficient for minimizing tagger errors. |
| title | Improving ML Training Data with Gold-Standard Quality Metrics |
| topic | Machine Learning I.2.7; I.2.6; G.3 |
| url | https://arxiv.org/abs/2512.20577 |