Data Virtualization for Machine Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915500200558592 |
|---|---|
| author | Khan, Saiful Chakraborty, Joyraj Beaucamp, Philip Bhujel, Niraj Chen, Min |
| author_facet | Khan, Saiful Chakraborty, Joyraj Beaucamp, Philip Bhujel, Niraj Chen, Min |
| contents | Nowadays, machine learning (ML) teams have multiple concurrent ML workflows for different applications. Each workflow typically involves many experiments, iterations, and collaborative activities and commonly takes months and sometimes years from initial data wrangling to model deployment. Organizationally, there is a large amount of intermediate data to be stored, processed, and maintained. \emph{Data virtualization} becomes a critical technology in an infrastructure to serve ML workflows. In this paper, we present the design and implementation of a data virtualization service, focusing on its service architecture and service operations. The infrastructure currently supports six ML applications, each with more than one ML workflow. The data virtualization service allows the number of applications and workflows to grow in the coming years. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_17293 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Data Virtualization for Machine Learning Khan, Saiful Chakraborty, Joyraj Beaucamp, Philip Bhujel, Niraj Chen, Min Software Engineering Machine Learning Nowadays, machine learning (ML) teams have multiple concurrent ML workflows for different applications. Each workflow typically involves many experiments, iterations, and collaborative activities and commonly takes months and sometimes years from initial data wrangling to model deployment. Organizationally, there is a large amount of intermediate data to be stored, processed, and maintained. \emph{Data virtualization} becomes a critical technology in an infrastructure to serve ML workflows. In this paper, we present the design and implementation of a data virtualization service, focusing on its service architecture and service operations. The infrastructure currently supports six ML applications, each with more than one ML workflow. The data virtualization service allows the number of applications and workflows to grow in the coming years. |
| title | Data Virtualization for Machine Learning |
| topic | Software Engineering Machine Learning |
| url | https://arxiv.org/abs/2507.17293 |