Bayesian Inference for Correlated Human Experts and Classifiers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909640896282624 |
|---|---|
| author | Kelly, Markelle Boyd, Alex Showalter, Sam Steyvers, Mark Smyth, Padhraic |
| author_facet | Kelly, Markelle Boyd, Alex Showalter, Sam Steyvers, Mark Smyth, Padhraic |
| contents | Applications of machine learning often involve making predictions based on both model outputs and the opinions of human experts. In this context, we investigate the problem of querying experts for class label predictions, using as few human queries as possible, and leveraging the class probability estimates of pre-trained classifiers. We develop a general Bayesian framework for this problem, modeling expert correlation via a joint latent representation, enabling simulation-based inference about the utility of additional expert queries, as well as inference of posterior distributions over unobserved expert labels. We apply our approach to two real-world medical classification problems, as well as to CIFAR-10H and ImageNet-16H, demonstrating substantial reductions relative to baselines in the cost of querying human experts while maintaining high prediction accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_05636 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bayesian Inference for Correlated Human Experts and Classifiers Kelly, Markelle Boyd, Alex Showalter, Sam Steyvers, Mark Smyth, Padhraic Machine Learning Artificial Intelligence Applications of machine learning often involve making predictions based on both model outputs and the opinions of human experts. In this context, we investigate the problem of querying experts for class label predictions, using as few human queries as possible, and leveraging the class probability estimates of pre-trained classifiers. We develop a general Bayesian framework for this problem, modeling expert correlation via a joint latent representation, enabling simulation-based inference about the utility of additional expert queries, as well as inference of posterior distributions over unobserved expert labels. We apply our approach to two real-world medical classification problems, as well as to CIFAR-10H and ImageNet-16H, demonstrating substantial reductions relative to baselines in the cost of querying human experts while maintaining high prediction accuracy. |
| title | Bayesian Inference for Correlated Human Experts and Classifiers |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.05636 |