Making LLMs Work for Enterprise Data Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917736444067840 |
|---|---|
| author | Demiralp, Çağatay Wenz, Fabian Chen, Peter Baile Kayali, Moe Tatbul, Nesime Stonebraker, Michael |
| author_facet | Demiralp, Çağatay Wenz, Fabian Chen, Peter Baile Kayali, Moe Tatbul, Nesime Stonebraker, Michael |
| contents | Large language models (LLMs) know little about enterprise database tables in the private data ecosystem, which substantially differ from web text in structure and content. As LLMs' performance is tied to their training data, a crucial question is how useful they can be in improving enterprise database management and analysis tasks. To address this, we contribute experimental results on LLMs' performance for text-to-SQL and semantic column-type detection tasks on enterprise datasets. The performance of LLMs on enterprise data is significantly lower than on benchmark datasets commonly used. Informed by our findings and feedback from industry practitioners, we identify three fundamental challenges -- latency, cost, and quality -- and propose potential solutions to use LLMs in enterprise data workflows effectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_20256 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Making LLMs Work for Enterprise Data Tasks Demiralp, Çağatay Wenz, Fabian Chen, Peter Baile Kayali, Moe Tatbul, Nesime Stonebraker, Michael Databases Artificial Intelligence Machine Learning Large language models (LLMs) know little about enterprise database tables in the private data ecosystem, which substantially differ from web text in structure and content. As LLMs' performance is tied to their training data, a crucial question is how useful they can be in improving enterprise database management and analysis tasks. To address this, we contribute experimental results on LLMs' performance for text-to-SQL and semantic column-type detection tasks on enterprise datasets. The performance of LLMs on enterprise data is significantly lower than on benchmark datasets commonly used. Informed by our findings and feedback from industry practitioners, we identify three fundamental challenges -- latency, cost, and quality -- and propose potential solutions to use LLMs in enterprise data workflows effectively. |
| title | Making LLMs Work for Enterprise Data Tasks |
| topic | Databases Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.20256 |