Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2405.06034 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913345742831616 |
|---|---|
| author | Hofer, R. Alex Maynez, Joshua Dhingra, Bhuwan Fisch, Adam Globerson, Amir Cohen, William W. |
| author_facet | Hofer, R. Alex Maynez, Joshua Dhingra, Bhuwan Fisch, Adam Globerson, Amir Cohen, William W. |
| contents | Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. Specifically, PPI methods provide tighter confidence intervals by combining small amounts of human-labeled data with larger amounts of data labeled by a reasonably accurate, but potentially biased, automatic system. We propose a framework for PPI based on Bayesian inference that allows researchers to develop new task-appropriate PPI methods easily. Exploiting the ease with which we can design new metrics, we propose improved PPI methods for several importantcases, such as autoraters that give discrete responses (e.g., prompted LLM ``judges'') and autoraters with scores that have a non-linear relationship to human scores. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_06034 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bayesian Prediction-Powered Inference Hofer, R. Alex Maynez, Joshua Dhingra, Bhuwan Fisch, Adam Globerson, Amir Cohen, William W. Machine Learning Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. Specifically, PPI methods provide tighter confidence intervals by combining small amounts of human-labeled data with larger amounts of data labeled by a reasonably accurate, but potentially biased, automatic system. We propose a framework for PPI based on Bayesian inference that allows researchers to develop new task-appropriate PPI methods easily. Exploiting the ease with which we can design new metrics, we propose improved PPI methods for several importantcases, such as autoraters that give discrete responses (e.g., prompted LLM ``judges'') and autoraters with scores that have a non-linear relationship to human scores. |
| title | Bayesian Prediction-Powered Inference |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2405.06034 |