Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910646057041920 |
|---|---|
| author | Fang, Chenhao Larson, Derek Zhu, Shitong Zeng, Sophie Summer, Wendy Peng, Yanqing Hulovatyy, Yuriy Rao, Rajeev Forgues, Gabriel Pudota, Arya Goncalves, Alex Robert, Hervé |
| author_facet | Fang, Chenhao Larson, Derek Zhu, Shitong Zeng, Sophie Summer, Wendy Peng, Yanqing Hulovatyy, Yuriy Rao, Rajeev Forgues, Gabriel Pudota, Arya Goncalves, Alex Robert, Hervé |
| contents | This paper presents new methods that have the potential to improve privacy process efficiency with LLM and RAG. To reduce hallucination, we continually pre-train the base LLM model with a privacy-specific knowledge base and then augment it with a semantic RAG layer. Our evaluations demonstrate that this approach enhances the model performance (as much as doubled metrics compared to out-of-box LLM) in handling privacy-related queries, by grounding responses with factual information which reduces inaccuracies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02825 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG Fang, Chenhao Larson, Derek Zhu, Shitong Zeng, Sophie Summer, Wendy Peng, Yanqing Hulovatyy, Yuriy Rao, Rajeev Forgues, Gabriel Pudota, Arya Goncalves, Alex Robert, Hervé Computation and Language Cryptography and Security This paper presents new methods that have the potential to improve privacy process efficiency with LLM and RAG. To reduce hallucination, we continually pre-train the base LLM model with a privacy-specific knowledge base and then augment it with a semantic RAG layer. Our evaluations demonstrate that this approach enhances the model performance (as much as doubled metrics compared to out-of-box LLM) in handling privacy-related queries, by grounding responses with factual information which reduces inaccuracies. |
| title | Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG |
| topic | Computation and Language Cryptography and Security |
| url | https://arxiv.org/abs/2410.02825 |