Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.07694 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912701109764096 |
|---|---|
| author | Nguyen, Manh Gupta, Sunil Le, Hung |
| author_facet | Nguyen, Manh Gupta, Sunil Le, Hung |
| contents | Large Language Models (LLMs) exhibit strong performance across various natural language processing (NLP) tasks but remain vulnerable to hallucinations, generating factually incorrect or misleading outputs. Uncertainty estimation, often using predictive entropy estimation, is key to addressing this issue. However, existing methods often require multiple samples or extra computation to assess semantic entropy. This paper proposes an efficient, training-free uncertainty estimation method that approximates predictive entropy using the responses' top-$K$ probabilities. Moreover, we employ an adaptive mechanism to determine $K$ to enhance flexibility and filter out low-confidence probabilities. Experimental results on three free-form question-answering datasets across several LLMs demonstrate that our method outperforms expensive state-of-the-art baselines, contributing to the broader goal of enhancing LLM trustworthiness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_07694 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Probabilities Are All You Need: A Probability-Only Approach to Uncertainty Estimation in Large Language Models Nguyen, Manh Gupta, Sunil Le, Hung Machine Learning Large Language Models (LLMs) exhibit strong performance across various natural language processing (NLP) tasks but remain vulnerable to hallucinations, generating factually incorrect or misleading outputs. Uncertainty estimation, often using predictive entropy estimation, is key to addressing this issue. However, existing methods often require multiple samples or extra computation to assess semantic entropy. This paper proposes an efficient, training-free uncertainty estimation method that approximates predictive entropy using the responses' top-$K$ probabilities. Moreover, we employ an adaptive mechanism to determine $K$ to enhance flexibility and filter out low-confidence probabilities. Experimental results on three free-form question-answering datasets across several LLMs demonstrate that our method outperforms expensive state-of-the-art baselines, contributing to the broader goal of enhancing LLM trustworthiness. |
| title | Probabilities Are All You Need: A Probability-Only Approach to Uncertainty Estimation in Large Language Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2511.07694 |