_version_ 1866929334704406528
author Zhu, Yiwen
Tian, Yuanyuan
Cahoon, Joyce
Krishnan, Subru
Agarwal, Ankita
Alotaibi, Rana
Camacho-Rodríguez, Jesús
Chundatt, Bibin
Chung, Andrew
Dutta, Niharika
Fogarty, Andrew
Gruenheid, Anja
Haynes, Brandon
Interlandi, Matteo
Iyer, Minu
Jurgens, Nick
Khushalani, Sumeet
Kroth, Brian
Kumar, Manoj
Leeka, Jyoti
Matusevych, Sergiy
Mittal, Minni
Mueller, Andreas
Muthyala, Kartheek
Nagulapalli, Harsha
Park, Yoonjae
Patel, Hiren
Pavlenko, Anna
Poppe, Olga
Ravindran, Santhosh
Saur, Karla
Sen, Rathijit
Suh, Steve
Tarafdar, Arijit
Waghray, Kunal
Wang, Demin
Curino, Carlo
Ramakrishnan, Raghu
author_facet Zhu, Yiwen
Tian, Yuanyuan
Cahoon, Joyce
Krishnan, Subru
Agarwal, Ankita
Alotaibi, Rana
Camacho-Rodríguez, Jesús
Chundatt, Bibin
Chung, Andrew
Dutta, Niharika
Fogarty, Andrew
Gruenheid, Anja
Haynes, Brandon
Interlandi, Matteo
Iyer, Minu
Jurgens, Nick
Khushalani, Sumeet
Kroth, Brian
Kumar, Manoj
Leeka, Jyoti
Matusevych, Sergiy
Mittal, Minni
Mueller, Andreas
Muthyala, Kartheek
Nagulapalli, Harsha
Park, Yoonjae
Patel, Hiren
Pavlenko, Anna
Poppe, Olga
Ravindran, Santhosh
Saur, Karla
Sen, Rathijit
Suh, Steve
Tarafdar, Arijit
Waghray, Kunal
Wang, Demin
Curino, Carlo
Ramakrishnan, Raghu
contents Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. However, the cloud brings in both complexity and opportunity. While cloud users can quickly start an application by using various data services, it can be difficult to configure and optimize these services to gain the most value from them. For cloud providers, managing every aspect of an ever-increasing set of data services, while meeting customer SLAs and minimizing operational cost is becoming more challenging. Cloud technology enables the collection of significant amounts of workload traces and system telemetry. With the progress in data science (DS) and machine learning (ML), it is feasible and desirable to utilize a data-driven, ML-based approach to automate various aspects of data services, resulting in the creation of autonomous data services. This paper presents our perspectives and insights on creating autonomous data services on Azure. It also covers the future endeavors we plan to undertake and unresolved issues that still need attention.
format Preprint
id arxiv_https___arxiv_org_abs_2405_01813
institution arXiv
publishDate 2024
record_format arxiv
spellingShingle Towards Building Autonomous Data Services on Azure
Zhu, Yiwen
Tian, Yuanyuan
Cahoon, Joyce
Krishnan, Subru
Agarwal, Ankita
Alotaibi, Rana
Camacho-Rodríguez, Jesús
Chundatt, Bibin
Chung, Andrew
Dutta, Niharika
Fogarty, Andrew
Gruenheid, Anja
Haynes, Brandon
Interlandi, Matteo
Iyer, Minu
Jurgens, Nick
Khushalani, Sumeet
Kroth, Brian
Kumar, Manoj
Leeka, Jyoti
Matusevych, Sergiy
Mittal, Minni
Mueller, Andreas
Muthyala, Kartheek
Nagulapalli, Harsha
Park, Yoonjae
Patel, Hiren
Pavlenko, Anna
Poppe, Olga
Ravindran, Santhosh
Saur, Karla
Sen, Rathijit
Suh, Steve
Tarafdar, Arijit
Waghray, Kunal
Wang, Demin
Curino, Carlo
Ramakrishnan, Raghu
Distributed, Parallel, and Cluster Computing
Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. However, the cloud brings in both complexity and opportunity. While cloud users can quickly start an application by using various data services, it can be difficult to configure and optimize these services to gain the most value from them. For cloud providers, managing every aspect of an ever-increasing set of data services, while meeting customer SLAs and minimizing operational cost is becoming more challenging. Cloud technology enables the collection of significant amounts of workload traces and system telemetry. With the progress in data science (DS) and machine learning (ML), it is feasible and desirable to utilize a data-driven, ML-based approach to automate various aspects of data services, resulting in the creation of autonomous data services. This paper presents our perspectives and insights on creating autonomous data services on Azure. It also covers the future endeavors we plan to undertake and unresolved issues that still need attention.
title Towards Building Autonomous Data Services on Azure
topic Distributed, Parallel, and Cluster Computing
url https://arxiv.org/abs/2405.01813