Confidence-weighted integration of human and machine judgments for superior decision-making
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908586024632320 |
|---|---|
| author | Yáñez, Felipe Luo, Xiaoliang Minero, Omar Valerio Love, Bradley C. |
| author_facet | Yáñez, Felipe Luo, Xiaoliang Minero, Omar Valerio Love, Bradley C. |
| contents | Large language models (LLMs) can surpass humans in certain forecasting tasks. What role does this leave for humans in the overall decision process? One possibility is that humans, despite performing worse than LLMs, can still add value when teamed with them. A human and machine team can surpass each individual teammate when team members' confidence is well-calibrated and team members diverge in which tasks they find difficult (i.e., calibration and diversity are needed). We simplified and extended a Bayesian approach to combining judgments using a logistic regression framework that integrates confidence-weighted judgments for any number of team members. Using this straightforward method, we demonstrated its effectiveness in both image classification and neuroscience forecasting tasks. Combining human judgments with one or more machines consistently improved overall team performance. Our hope is that this simple and effective strategy for integrating the judgments of humans and machines will lead to productive collaborations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_08083 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Confidence-weighted integration of human and machine judgments for superior decision-making Yáñez, Felipe Luo, Xiaoliang Minero, Omar Valerio Love, Bradley C. Human-Computer Interaction Artificial Intelligence Neurons and Cognition Large language models (LLMs) can surpass humans in certain forecasting tasks. What role does this leave for humans in the overall decision process? One possibility is that humans, despite performing worse than LLMs, can still add value when teamed with them. A human and machine team can surpass each individual teammate when team members' confidence is well-calibrated and team members diverge in which tasks they find difficult (i.e., calibration and diversity are needed). We simplified and extended a Bayesian approach to combining judgments using a logistic regression framework that integrates confidence-weighted judgments for any number of team members. Using this straightforward method, we demonstrated its effectiveness in both image classification and neuroscience forecasting tasks. Combining human judgments with one or more machines consistently improved overall team performance. Our hope is that this simple and effective strategy for integrating the judgments of humans and machines will lead to productive collaborations. |
| title | Confidence-weighted integration of human and machine judgments for superior decision-making |
| topic | Human-Computer Interaction Artificial Intelligence Neurons and Cognition |
| url | https://arxiv.org/abs/2408.08083 |