Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.12748 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914921345712128 |
|---|---|
| author | Hu, Mengya Xu, Rui Lei, Deren Li, Yaxi Wang, Mingyu Ching, Emily Kamal, Eslam Deng, Alex |
| author_facet | Hu, Mengya Xu, Rui Lei, Deren Li, Yaxi Wang, Mingyu Ching, Emily Kamal, Eslam Deng, Alex |
| contents | Large language models (LLMs) are highly capable but face latency challenges in real-time applications, such as conducting online hallucination detection. To overcome this issue, we propose a novel framework that leverages a small language model (SLM) classifier for initial detection, followed by a LLM as constrained reasoner to generate detailed explanations for detected hallucinated content. This study optimizes the real-time interpretable hallucination detection by introducing effective prompting techniques that align LLM-generated explanations with SLM decisions. Empirical experiment results demonstrate its effectiveness, thereby enhancing the overall user experience. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_12748 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection Hu, Mengya Xu, Rui Lei, Deren Li, Yaxi Wang, Mingyu Ching, Emily Kamal, Eslam Deng, Alex Computation and Language Artificial Intelligence Machine Learning Large language models (LLMs) are highly capable but face latency challenges in real-time applications, such as conducting online hallucination detection. To overcome this issue, we propose a novel framework that leverages a small language model (SLM) classifier for initial detection, followed by a LLM as constrained reasoner to generate detailed explanations for detected hallucinated content. This study optimizes the real-time interpretable hallucination detection by introducing effective prompting techniques that align LLM-generated explanations with SLM decisions. Empirical experiment results demonstrate its effectiveness, thereby enhancing the overall user experience. |
| title | SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2408.12748 |