The Design of an LLM-powered Unstructured Analytics System
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909442964979712 |
|---|---|
| author | Anderson, Eric Fritz, Jonathan Lee, Austin Li, Bohou Lindblad, Mark Lindeman, Henry Meyer, Alex Parmar, Parth Ranade, Tanvi Shah, Mehul A. Sowell, Benjamin Tecuci, Dan Thapliyal, Vinayak Welsh, Matt |
| author_facet | Anderson, Eric Fritz, Jonathan Lee, Austin Li, Bohou Lindblad, Mark Lindeman, Henry Meyer, Alex Parmar, Parth Ranade, Tanvi Shah, Mehul A. Sowell, Benjamin Tecuci, Dan Thapliyal, Vinayak Welsh, Matt |
| contents | LLMs demonstrate an uncanny ability to process unstructured data, and as such, have the potential to go beyond search and run complex, semantic analyses at scale. We describe the design of an unstructured analytics system, Aryn, and the tenets and use cases that motivate its design. With Aryn, users specify queries in natural language and the system automatically determines a semantic plan and executes it to compute an answer from a large collection of unstructured documents. At the core of Aryn is Sycamore, a declarative document processing engine, that provides a reliable distributed abstraction called DocSets. Sycamore allows users to analyze, enrich, and transform complex documents at scale. Aryn includes Luna, a query planner that translates natural language queries to Sycamore scripts, and DocParse, which takes raw PDFs and document images, and converts them to DocSets for downstream processing. We show how these pieces come together to achieve better accuracy than RAG on analytics queries over real world reports from the National Transportation Safety Board (NTSB). Also, given current limitations of LLMs, we argue that an analytics system must provide explainability to be practical, and show how Aryn's user interface does this to help build trust. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_00847 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Design of an LLM-powered Unstructured Analytics System Anderson, Eric Fritz, Jonathan Lee, Austin Li, Bohou Lindblad, Mark Lindeman, Henry Meyer, Alex Parmar, Parth Ranade, Tanvi Shah, Mehul A. Sowell, Benjamin Tecuci, Dan Thapliyal, Vinayak Welsh, Matt Databases Artificial Intelligence Information Retrieval LLMs demonstrate an uncanny ability to process unstructured data, and as such, have the potential to go beyond search and run complex, semantic analyses at scale. We describe the design of an unstructured analytics system, Aryn, and the tenets and use cases that motivate its design. With Aryn, users specify queries in natural language and the system automatically determines a semantic plan and executes it to compute an answer from a large collection of unstructured documents. At the core of Aryn is Sycamore, a declarative document processing engine, that provides a reliable distributed abstraction called DocSets. Sycamore allows users to analyze, enrich, and transform complex documents at scale. Aryn includes Luna, a query planner that translates natural language queries to Sycamore scripts, and DocParse, which takes raw PDFs and document images, and converts them to DocSets for downstream processing. We show how these pieces come together to achieve better accuracy than RAG on analytics queries over real world reports from the National Transportation Safety Board (NTSB). Also, given current limitations of LLMs, we argue that an analytics system must provide explainability to be practical, and show how Aryn's user interface does this to help build trust. |
| title | The Design of an LLM-powered Unstructured Analytics System |
| topic | Databases Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2409.00847 |