Estimating the Hallucination Rate of Generative AI
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908593039605760 |
|---|---|
| author | Jesson, Andrew Beltran-Velez, Nicolas Chu, Quentin Karlekar, Sweta Kossen, Jannik Gal, Yarin Cunningham, John P. Blei, David |
| author_facet | Jesson, Andrew Beltran-Velez, Nicolas Chu, Quentin Karlekar, Sweta Kossen, Jannik Gal, Yarin Cunningham, John P. Blei, David |
| contents | This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07457 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Estimating the Hallucination Rate of Generative AI Jesson, Andrew Beltran-Velez, Nicolas Chu, Quentin Karlekar, Sweta Kossen, Jannik Gal, Yarin Cunningham, John P. Blei, David Machine Learning This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks. |
| title | Estimating the Hallucination Rate of Generative AI |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.07457 |