Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909859174154240 |
|---|---|
| author | Feng, Yu Htut, Phu Mon Qi, Zheng Xiao, Wei Mager, Manuel Pappas, Nikolaos Halder, Kishaloy Li, Yang Benajiba, Yassine Roth, Dan |
| author_facet | Feng, Yu Htut, Phu Mon Qi, Zheng Xiao, Wei Mager, Manuel Pappas, Nikolaos Halder, Kishaloy Li, Yang Benajiba, Yassine Roth, Dan |
| contents | Quantifying uncertainty in black-box LLMs is vital for reliable responses and scalable oversight. Existing methods, which gauge a model's uncertainty through evaluating self-consistency in responses to the target query, can be misleading: an LLM may confidently provide an incorrect answer to a target query, yet give a confident and accurate answer to that same target query when answering a knowledge-preserving perturbation of the query. We systematically analyze the model behaviors and demonstrate that this discrepancy stems from suboptimal retrieval of parametric knowledge, often due to contextual biases that prevent consistent access to stored knowledge. We then introduce DiverseAgentEntropy, a novel, theoretically-grounded method employing multi-agent interaction across diverse query variations for uncertainty estimation of black-box LLMs. This approach more accurately assesses an LLM's true uncertainty and improves hallucination detection, outperforming existing self-consistency based techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_09572 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty Feng, Yu Htut, Phu Mon Qi, Zheng Xiao, Wei Mager, Manuel Pappas, Nikolaos Halder, Kishaloy Li, Yang Benajiba, Yassine Roth, Dan Computation and Language Quantifying uncertainty in black-box LLMs is vital for reliable responses and scalable oversight. Existing methods, which gauge a model's uncertainty through evaluating self-consistency in responses to the target query, can be misleading: an LLM may confidently provide an incorrect answer to a target query, yet give a confident and accurate answer to that same target query when answering a knowledge-preserving perturbation of the query. We systematically analyze the model behaviors and demonstrate that this discrepancy stems from suboptimal retrieval of parametric knowledge, often due to contextual biases that prevent consistent access to stored knowledge. We then introduce DiverseAgentEntropy, a novel, theoretically-grounded method employing multi-agent interaction across diverse query variations for uncertainty estimation of black-box LLMs. This approach more accurately assesses an LLM's true uncertainty and improves hallucination detection, outperforming existing self-consistency based techniques. |
| title | Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.09572 |