Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.22396 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909976740495360 |
|---|---|
| author | Vangala, Bhanu Prakash Mahmud, Sajid Neupane, Pawan Selvaraj, Joel Cheng, Jianlin |
| author_facet | Vangala, Bhanu Prakash Mahmud, Sajid Neupane, Pawan Selvaraj, Joel Cheng, Jianlin |
| contents | Artificial Intelligence (AI), particularly Large Language Models (LLMs), is transforming scientific discovery, enabling rapid knowledge generation and hypothesis formulation. However, a critical challenge is hallucination, where LLMs generate factually incorrect or misleading information, compromising research integrity. To address this, we introduce HalluMatData, a benchmark dataset for evaluating hallucination detection methods, factual consistency, and response robustness in AI-generated materials science content. Alongside this, we propose HalluMatDetector, a multi-stage hallucination detection framework that integrates intrinsic verification, multi-source retrieval, contradiction graph analysis, and metric-based assessment to detect and mitigate LLM hallucinations. Our findings reveal that hallucination levels vary significantly across materials science subdomains, with high-entropy queries exhibiting greater factual inconsistencies. By utilizing HalluMatDetector verification pipeline, we reduce hallucination rates by 30% compared to standard LLM outputs. Furthermore, we introduce the Paraphrased Hallucination Consistency Score (PHCS) to quantify inconsistencies in LLM responses across semantically equivalent queries, offering deeper insights into model reliability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_22396 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HalluMat: Detecting Hallucinations in LLM-Generated Materials Science Content Through Multi-Stage Verification Vangala, Bhanu Prakash Mahmud, Sajid Neupane, Pawan Selvaraj, Joel Cheng, Jianlin Artificial Intelligence Materials Science Information Retrieval Artificial Intelligence (AI), particularly Large Language Models (LLMs), is transforming scientific discovery, enabling rapid knowledge generation and hypothesis formulation. However, a critical challenge is hallucination, where LLMs generate factually incorrect or misleading information, compromising research integrity. To address this, we introduce HalluMatData, a benchmark dataset for evaluating hallucination detection methods, factual consistency, and response robustness in AI-generated materials science content. Alongside this, we propose HalluMatDetector, a multi-stage hallucination detection framework that integrates intrinsic verification, multi-source retrieval, contradiction graph analysis, and metric-based assessment to detect and mitigate LLM hallucinations. Our findings reveal that hallucination levels vary significantly across materials science subdomains, with high-entropy queries exhibiting greater factual inconsistencies. By utilizing HalluMatDetector verification pipeline, we reduce hallucination rates by 30% compared to standard LLM outputs. Furthermore, we introduce the Paraphrased Hallucination Consistency Score (PHCS) to quantify inconsistencies in LLM responses across semantically equivalent queries, offering deeper insights into model reliability. |
| title | HalluMat: Detecting Hallucinations in LLM-Generated Materials Science Content Through Multi-Stage Verification |
| topic | Artificial Intelligence Materials Science Information Retrieval |
| url | https://arxiv.org/abs/2512.22396 |