On the Role of Unobserved Sequences on Sample-based Uncertainty Quantification for LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909825937440768 |
|---|---|
| author | Kunitomo-Jacquin, Lucie Marrese-Taylor, Edison Fukuda, Ken |
| author_facet | Kunitomo-Jacquin, Lucie Marrese-Taylor, Edison Fukuda, Ken |
| contents | Quantifying uncertainty in large language models (LLMs) is important for safety-critical applications because it helps spot incorrect answers, known as hallucinations. One major trend of uncertainty quantification methods is based on estimating the entropy of the distribution of the LLM's potential output sequences. This estimation is based on a set of output sequences and associated probabilities obtained by querying the LLM several times. In this paper, we advocate and experimentally show that the probability of unobserved sequences plays a crucial role, and we recommend future research to integrate it to enhance such LLM uncertainty quantification methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04439 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Role of Unobserved Sequences on Sample-based Uncertainty Quantification for LLMs Kunitomo-Jacquin, Lucie Marrese-Taylor, Edison Fukuda, Ken Computation and Language Quantifying uncertainty in large language models (LLMs) is important for safety-critical applications because it helps spot incorrect answers, known as hallucinations. One major trend of uncertainty quantification methods is based on estimating the entropy of the distribution of the LLM's potential output sequences. This estimation is based on a set of output sequences and associated probabilities obtained by querying the LLM several times. In this paper, we advocate and experimentally show that the probability of unobserved sequences plays a crucial role, and we recommend future research to integrate it to enhance such LLM uncertainty quantification methods. |
| title | On the Role of Unobserved Sequences on Sample-based Uncertainty Quantification for LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.04439 |