Lynx: An Open Source Hallucination Evaluation Model
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910538790862848 |
|---|---|
| author | Ravi, Selvan Sunitha Mielczarek, Bartosz Kannappan, Anand Kiela, Douwe Qian, Rebecca |
| author_facet | Ravi, Selvan Sunitha Mielczarek, Bartosz Kannappan, Anand Kiela, Douwe Qian, Rebecca |
| contents | Retrieval Augmented Generation (RAG) techniques aim to mitigate hallucinations in Large Language Models (LLMs). However, LLMs can still produce information that is unsupported or contradictory to the retrieved contexts. We introduce LYNX, a SOTA hallucination detection LLM that is capable of advanced reasoning on challenging real-world hallucination scenarios. To evaluate LYNX, we present HaluBench, a comprehensive hallucination evaluation benchmark, consisting of 15k samples sourced from various real-world domains. Our experiment results show that LYNX outperforms GPT-4o, Claude-3-Sonnet, and closed and open-source LLM-as-a-judge models on HaluBench. We release LYNX, HaluBench and our evaluation code for public access. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08488 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lynx: An Open Source Hallucination Evaluation Model Ravi, Selvan Sunitha Mielczarek, Bartosz Kannappan, Anand Kiela, Douwe Qian, Rebecca Artificial Intelligence Computation and Language Retrieval Augmented Generation (RAG) techniques aim to mitigate hallucinations in Large Language Models (LLMs). However, LLMs can still produce information that is unsupported or contradictory to the retrieved contexts. We introduce LYNX, a SOTA hallucination detection LLM that is capable of advanced reasoning on challenging real-world hallucination scenarios. To evaluate LYNX, we present HaluBench, a comprehensive hallucination evaluation benchmark, consisting of 15k samples sourced from various real-world domains. Our experiment results show that LYNX outperforms GPT-4o, Claude-3-Sonnet, and closed and open-source LLM-as-a-judge models on HaluBench. We release LYNX, HaluBench and our evaluation code for public access. |
| title | Lynx: An Open Source Hallucination Evaluation Model |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2407.08488 |