Introducing Schema Inference as a Scalable SQL Function [Extended Version]
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913581908361216 |
|---|---|
| author | Dani, Calvin Jahangiri, Shiva Hütter, Thomas |
| author_facet | Dani, Calvin Jahangiri, Shiva Hütter, Thomas |
| contents | This paper introduces a novel approach to schema inference as an on-demand function integrated directly within a DBMS, targeting NoSQL databases where schema flexibility can create challenges. Unlike previous methods relying on external frameworks like Apache Spark, our solution enables schema inference as a SQL function, allowing users to infer schemas natively within the DBMS. Implemented in Apache AsterixDB, it performs schema discovery in two phases, local inference and global schema merging, leveraging internal resources for improved performance. Experiments with real world datasets show up to a two orders of magnitude performance boost over external methods, enhancing usability and scalability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_13278 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Introducing Schema Inference as a Scalable SQL Function [Extended Version] Dani, Calvin Jahangiri, Shiva Hütter, Thomas Databases This paper introduces a novel approach to schema inference as an on-demand function integrated directly within a DBMS, targeting NoSQL databases where schema flexibility can create challenges. Unlike previous methods relying on external frameworks like Apache Spark, our solution enables schema inference as a SQL function, allowing users to infer schemas natively within the DBMS. Implemented in Apache AsterixDB, it performs schema discovery in two phases, local inference and global schema merging, leveraging internal resources for improved performance. Experiments with real world datasets show up to a two orders of magnitude performance boost over external methods, enhancing usability and scalability. |
| title | Introducing Schema Inference as a Scalable SQL Function [Extended Version] |
| topic | Databases |
| url | https://arxiv.org/abs/2411.13278 |