AutoEval Done Right: Using Synthetic Data for Model Evaluation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913177112936448 |
|---|---|
| author | Boyeau, Pierre Angelopoulos, Anastasios N. Yosef, Nir Malik, Jitendra Jordan, Michael I. |
| author_facet | Boyeau, Pierre Angelopoulos, Anastasios N. Yosef, Nir Malik, Jitendra Jordan, Michael I. |
| contents | The evaluation of machine learning models using human-labeled validation data can be expensive and time-consuming. AI-labeled synthetic data can be used to decrease the number of human annotations required for this purpose in a process called autoevaluation. We suggest efficient and statistically principled algorithms for this purpose that improve sample efficiency while remaining unbiased. These algorithms increase the effective human-labeled sample size by up to 50% on experiments with GPT-4. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07008 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AutoEval Done Right: Using Synthetic Data for Model Evaluation Boyeau, Pierre Angelopoulos, Anastasios N. Yosef, Nir Malik, Jitendra Jordan, Michael I. Machine Learning Artificial Intelligence Computation and Language Methodology The evaluation of machine learning models using human-labeled validation data can be expensive and time-consuming. AI-labeled synthetic data can be used to decrease the number of human annotations required for this purpose in a process called autoevaluation. We suggest efficient and statistically principled algorithms for this purpose that improve sample efficiency while remaining unbiased. These algorithms increase the effective human-labeled sample size by up to 50% on experiments with GPT-4. |
| title | AutoEval Done Right: Using Synthetic Data for Model Evaluation |
| topic | Machine Learning Artificial Intelligence Computation and Language Methodology |
| url | https://arxiv.org/abs/2403.07008 |