CHORUS: Foundation Models for Unified Data Discovery and Exploration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929303900389376 |
|---|---|
| author | Kayali, Moe Lykov, Anton Fountalis, Ilias Vasiloglou, Nikolaos Olteanu, Dan Suciu, Dan |
| author_facet | Kayali, Moe Lykov, Anton Fountalis, Ilias Vasiloglou, Nikolaos Olteanu, Dan Suciu, Dan |
| contents | We apply foundation models to data discovery and exploration tasks. Foundation models include large language models (LLMs) that show promising performance on a range of diverse tasks unrelated to their training. We show that these models are highly applicable to the data discovery and data exploration domain. When carefully used, they have superior capability on three representative tasks: table-class detection, column-type annotation and join-column prediction. On all three tasks, we show that a foundation-model-based approach outperforms the task-specific models and so the state of the art. Further, our approach often surpasses human-expert task performance. We investigate the fundamental characteristics of this approach including generalizability to several foundation models and the impact of non-determinism on the outputs. All in all, this suggests a future direction in which disparate data management tasks can be unified under foundation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_09610 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CHORUS: Foundation Models for Unified Data Discovery and Exploration Kayali, Moe Lykov, Anton Fountalis, Ilias Vasiloglou, Nikolaos Olteanu, Dan Suciu, Dan Databases Machine Learning We apply foundation models to data discovery and exploration tasks. Foundation models include large language models (LLMs) that show promising performance on a range of diverse tasks unrelated to their training. We show that these models are highly applicable to the data discovery and data exploration domain. When carefully used, they have superior capability on three representative tasks: table-class detection, column-type annotation and join-column prediction. On all three tasks, we show that a foundation-model-based approach outperforms the task-specific models and so the state of the art. Further, our approach often surpasses human-expert task performance. We investigate the fundamental characteristics of this approach including generalizability to several foundation models and the impact of non-determinism on the outputs. All in all, this suggests a future direction in which disparate data management tasks can be unified under foundation models. |
| title | CHORUS: Foundation Models for Unified Data Discovery and Exploration |
| topic | Databases Machine Learning |
| url | https://arxiv.org/abs/2306.09610 |