OAEI-LLM: A Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matching
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910004638908416 |
|---|---|
| author | Qiang, Zhangcheng Taylor, Kerry Wang, Weiqing Jiang, Jing |
| author_facet | Qiang, Zhangcheng Taylor, Kerry Wang, Weiqing Jiang, Jing |
| contents | Hallucinations of large language models (LLMs) commonly occur in domain-specific downstream tasks, with no exception in ontology matching (OM). The prevalence of using LLMs for OM raises the need for benchmarks to better understand LLM hallucinations. The OAEI-LLM dataset is an extended version of the Ontology Alignment Evaluation Initiative (OAEI) datasets that evaluate LLM-specific hallucinations in OM tasks. We outline the methodology used in dataset construction and schema extension, and provide examples of potential use cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OAEI-LLM: A Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matching Qiang, Zhangcheng Taylor, Kerry Wang, Weiqing Jiang, Jing Artificial Intelligence Computation and Language Information Retrieval Hallucinations of large language models (LLMs) commonly occur in domain-specific downstream tasks, with no exception in ontology matching (OM). The prevalence of using LLMs for OM raises the need for benchmarks to better understand LLM hallucinations. The OAEI-LLM dataset is an extended version of the Ontology Alignment Evaluation Initiative (OAEI) datasets that evaluate LLM-specific hallucinations in OM tasks. We outline the methodology used in dataset construction and schema extension, and provide examples of potential use cases. |
| title | OAEI-LLM: A Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matching |
| topic | Artificial Intelligence Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2409.14038 |