HalluVerse25: Fine-grained Multilingual Benchmark Dataset for LLM Hallucinations
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912268749373440 |
|---|---|
| author | Abdaljalil, Samir Kurban, Hasan Serpedin, Erchin |
| author_facet | Abdaljalil, Samir Kurban, Hasan Serpedin, Erchin |
| contents | Large Language Models (LLMs) are increasingly used in various contexts, yet remain prone to generating non-factual content, commonly referred to as "hallucinations". The literature categorizes hallucinations into several types, including entity-level, relation-level, and sentence-level hallucinations. However, existing hallucination datasets often fail to capture fine-grained hallucinations in multilingual settings. In this work, we introduce HalluVerse25, a multilingual LLM hallucination dataset that categorizes fine-grained hallucinations in English, Arabic, and Turkish. Our dataset construction pipeline uses an LLM to inject hallucinations into factual biographical sentences, followed by a rigorous human annotation process to ensure data quality. We evaluate several LLMs on HalluVerse25, providing valuable insights into how proprietary models perform in detecting LLM-generated hallucinations across different contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07833 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HalluVerse25: Fine-grained Multilingual Benchmark Dataset for LLM Hallucinations Abdaljalil, Samir Kurban, Hasan Serpedin, Erchin Computation and Language Artificial Intelligence Large Language Models (LLMs) are increasingly used in various contexts, yet remain prone to generating non-factual content, commonly referred to as "hallucinations". The literature categorizes hallucinations into several types, including entity-level, relation-level, and sentence-level hallucinations. However, existing hallucination datasets often fail to capture fine-grained hallucinations in multilingual settings. In this work, we introduce HalluVerse25, a multilingual LLM hallucination dataset that categorizes fine-grained hallucinations in English, Arabic, and Turkish. Our dataset construction pipeline uses an LLM to inject hallucinations into factual biographical sentences, followed by a rigorous human annotation process to ensure data quality. We evaluate several LLMs on HalluVerse25, providing valuable insights into how proprietary models perform in detecting LLM-generated hallucinations across different contexts. |
| title | HalluVerse25: Fine-grained Multilingual Benchmark Dataset for LLM Hallucinations |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.07833 |