Knowing When to Ask -- Bridging Large Language Models and Data
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910617332350976 |
|---|---|
| author | Radhakrishnan, Prashanth Chen, Jennifer Xu, Bo Ramaswami, Prem Pho, Hannah Olmos, Adriana Manyika, James Guha, R. V. |
| author_facet | Radhakrishnan, Prashanth Chen, Jennifer Xu, Bo Ramaswami, Prem Pho, Hannah Olmos, Adriana Manyika, James Guha, R. V. |
| contents | Large Language Models (LLMs) are prone to generating factually incorrect information when responding to queries that involve numerical and statistical data or other timely facts. In this paper, we present an approach for enhancing the accuracy of LLMs by integrating them with Data Commons, a vast, open-source repository of public statistics from trusted organizations like the United Nations (UN), Center for Disease Control and Prevention (CDC) and global census bureaus. We explore two primary methods: Retrieval Interleaved Generation (RIG), where the LLM is trained to produce natural language queries to retrieve data from Data Commons, and Retrieval Augmented Generation (RAG), where relevant data tables are fetched from Data Commons and used to augment the LLM's prompt. We evaluate these methods on a diverse set of queries, demonstrating their effectiveness in improving the factual accuracy of LLM outputs. Our work represents an early step towards building more trustworthy and reliable LLMs that are grounded in verifiable statistical data and capable of complex factual reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13741 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Knowing When to Ask -- Bridging Large Language Models and Data Radhakrishnan, Prashanth Chen, Jennifer Xu, Bo Ramaswami, Prem Pho, Hannah Olmos, Adriana Manyika, James Guha, R. V. Computation and Language Information Retrieval Large Language Models (LLMs) are prone to generating factually incorrect information when responding to queries that involve numerical and statistical data or other timely facts. In this paper, we present an approach for enhancing the accuracy of LLMs by integrating them with Data Commons, a vast, open-source repository of public statistics from trusted organizations like the United Nations (UN), Center for Disease Control and Prevention (CDC) and global census bureaus. We explore two primary methods: Retrieval Interleaved Generation (RIG), where the LLM is trained to produce natural language queries to retrieve data from Data Commons, and Retrieval Augmented Generation (RAG), where relevant data tables are fetched from Data Commons and used to augment the LLM's prompt. We evaluate these methods on a diverse set of queries, demonstrating their effectiveness in improving the factual accuracy of LLM outputs. Our work represents an early step towards building more trustworthy and reliable LLMs that are grounded in verifiable statistical data and capable of complex factual reasoning. |
| title | Knowing When to Ask -- Bridging Large Language Models and Data |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2409.13741 |