Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913885104111616 |
|---|---|
| author | Nguyen-Hien, T. Duy Ivanova, Desi R. Teh, Yee Whye Lee, Wee Sun |
| author_facet | Nguyen-Hien, T. Duy Ivanova, Desi R. Teh, Yee Whye Lee, Wee Sun |
| contents | Although large language models (LLMs) are highly interactive and extendable, current approaches to ensure reliability in deployments remain mostly limited to rejecting outputs with high uncertainty in order to avoid misinformation. This conservative strategy reflects the current lack of tools to systematically distinguish and respond to different sources of uncertainty. In this paper, we advocate for the adoption of Bayesian Modeling of Experiments -- a framework that provides a coherent foundation to reason about uncertainty and clarify the reducibility of uncertainty -- for managing and proactively addressing uncertainty that arises in LLM deployments. This framework enables LLMs and their users to take contextually appropriate steps, such as requesting clarification, retrieving external information, or refining inputs. By supporting active resolution rather than passive avoidance, it opens the door to more reliable, transparent, and broadly applicable LLM systems, particularly in high-stakes, real-world settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07448 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs Nguyen-Hien, T. Duy Ivanova, Desi R. Teh, Yee Whye Lee, Wee Sun Machine Learning Artificial Intelligence Although large language models (LLMs) are highly interactive and extendable, current approaches to ensure reliability in deployments remain mostly limited to rejecting outputs with high uncertainty in order to avoid misinformation. This conservative strategy reflects the current lack of tools to systematically distinguish and respond to different sources of uncertainty. In this paper, we advocate for the adoption of Bayesian Modeling of Experiments -- a framework that provides a coherent foundation to reason about uncertainty and clarify the reducibility of uncertainty -- for managing and proactively addressing uncertainty that arises in LLM deployments. This framework enables LLMs and their users to take contextually appropriate steps, such as requesting clarification, retrieving external information, or refining inputs. By supporting active resolution rather than passive avoidance, it opens the door to more reliable, transparent, and broadly applicable LLM systems, particularly in high-stakes, real-world settings. |
| title | Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.07448 |