Using AI Uncertainty Quantification to Improve Human Decision-Making
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916116737032192 |
|---|---|
| author | Marusich, Laura R. Bakdash, Jonathan Z. Zhou, Yan Kantarcioglu, Murat |
| author_facet | Marusich, Laura R. Bakdash, Jonathan Z. Zhou, Yan Kantarcioglu, Murat |
| contents | AI Uncertainty Quantification (UQ) has the potential to improve human decision-making beyond AI predictions alone by providing additional probabilistic information to users. The majority of past research on AI and human decision-making has concentrated on model explainability and interpretability, with little focus on understanding the potential impact of UQ on human decision-making. We evaluated the impact on human decision-making for instance-level UQ, calibrated using a strict scoring rule, in two online behavioral experiments. In the first experiment, our results showed that UQ was beneficial for decision-making performance compared to only AI predictions. In the second experiment, we found UQ had generalizable benefits for decision-making across a variety of representations for probabilistic information. These results indicate that implementing high quality, instance-level UQ for AI may improve decision-making with real systems compared to AI predictions alone. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_10852 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Using AI Uncertainty Quantification to Improve Human Decision-Making Marusich, Laura R. Bakdash, Jonathan Z. Zhou, Yan Kantarcioglu, Murat Artificial Intelligence Human-Computer Interaction AI Uncertainty Quantification (UQ) has the potential to improve human decision-making beyond AI predictions alone by providing additional probabilistic information to users. The majority of past research on AI and human decision-making has concentrated on model explainability and interpretability, with little focus on understanding the potential impact of UQ on human decision-making. We evaluated the impact on human decision-making for instance-level UQ, calibrated using a strict scoring rule, in two online behavioral experiments. In the first experiment, our results showed that UQ was beneficial for decision-making performance compared to only AI predictions. In the second experiment, we found UQ had generalizable benefits for decision-making across a variety of representations for probabilistic information. These results indicate that implementing high quality, instance-level UQ for AI may improve decision-making with real systems compared to AI predictions alone. |
| title | Using AI Uncertainty Quantification to Improve Human Decision-Making |
| topic | Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2309.10852 |