LLM Augmentations to support Analytical Reasoning over Multiple Documents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929603819339776 |
|---|---|
| author | Yousuf, Raquib Bin Defelice, Nicholas Sharma, Mandar Xu, Shengzhe Ramakrishnan, Naren |
| author_facet | Yousuf, Raquib Bin Defelice, Nicholas Sharma, Mandar Xu, Shengzhe Ramakrishnan, Naren |
| contents | Building on their demonstrated ability to perform a variety of tasks, we investigate the application of large language models (LLMs) to enhance in-depth analytical reasoning within the context of intelligence analysis. Intelligence analysts typically work with massive dossiers to draw connections between seemingly unrelated entities, and uncover adversaries' plans and motives. We explore if and how LLMs can be helpful to analysts for this task and develop an architecture to augment the capabilities of an LLM with a memory module called dynamic evidence trees (DETs) to develop and track multiple investigation threads. Through extensive experiments on multiple datasets, we highlight how LLMs, as-is, are still inadequate to support intelligence analysts and offer recommendations to improve LLMs for such intricate reasoning applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_16116 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLM Augmentations to support Analytical Reasoning over Multiple Documents Yousuf, Raquib Bin Defelice, Nicholas Sharma, Mandar Xu, Shengzhe Ramakrishnan, Naren Computation and Language Artificial Intelligence Building on their demonstrated ability to perform a variety of tasks, we investigate the application of large language models (LLMs) to enhance in-depth analytical reasoning within the context of intelligence analysis. Intelligence analysts typically work with massive dossiers to draw connections between seemingly unrelated entities, and uncover adversaries' plans and motives. We explore if and how LLMs can be helpful to analysts for this task and develop an architecture to augment the capabilities of an LLM with a memory module called dynamic evidence trees (DETs) to develop and track multiple investigation threads. Through extensive experiments on multiple datasets, we highlight how LLMs, as-is, are still inadequate to support intelligence analysts and offer recommendations to improve LLMs for such intricate reasoning applications. |
| title | LLM Augmentations to support Analytical Reasoning over Multiple Documents |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.16116 |