The Two Sides of the Coin: Hallucination Generation and Detection with LLMs as Evaluators for LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916320644169728 |
|---|---|
| author | Bui, Anh Thu Maria Brech, Saskia Felizitas Hußfeldt, Natalie Jennert, Tobias Ullrich, Melanie Breuer, Timo Khasmakhi, Narjes Nikzad Schaer, Philipp |
| author_facet | Bui, Anh Thu Maria Brech, Saskia Felizitas Hußfeldt, Natalie Jennert, Tobias Ullrich, Melanie Breuer, Timo Khasmakhi, Narjes Nikzad Schaer, Philipp |
| contents | Hallucination detection in Large Language Models (LLMs) is crucial for ensuring their reliability. This work presents our participation in the CLEF ELOQUENT HalluciGen shared task, where the goal is to develop evaluators for both generating and detecting hallucinated content. We explored the capabilities of four LLMs: Llama 3, Gemma, GPT-3.5 Turbo, and GPT-4, for this purpose. We also employed ensemble majority voting to incorporate all four models for the detection task. The results provide valuable insights into the strengths and weaknesses of these LLMs in handling hallucination generation and detection tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_09152 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Two Sides of the Coin: Hallucination Generation and Detection with LLMs as Evaluators for LLMs Bui, Anh Thu Maria Brech, Saskia Felizitas Hußfeldt, Natalie Jennert, Tobias Ullrich, Melanie Breuer, Timo Khasmakhi, Narjes Nikzad Schaer, Philipp Artificial Intelligence Computation and Language Hallucination detection in Large Language Models (LLMs) is crucial for ensuring their reliability. This work presents our participation in the CLEF ELOQUENT HalluciGen shared task, where the goal is to develop evaluators for both generating and detecting hallucinated content. We explored the capabilities of four LLMs: Llama 3, Gemma, GPT-3.5 Turbo, and GPT-4, for this purpose. We also employed ensemble majority voting to incorporate all four models for the detection task. The results provide valuable insights into the strengths and weaknesses of these LLMs in handling hallucination generation and detection tasks. |
| title | The Two Sides of the Coin: Hallucination Generation and Detection with LLMs as Evaluators for LLMs |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2407.09152 |