An Empirical Study of Production Incidents in Generative AI Cloud Services
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909736320892928 |
|---|---|
| author | Yan, Haoran Chen, Yinfang Ma, Minghua Wen, Ming Lu, Shan Zhang, Shenglin Xu, Tianyin Wang, Rujia Bansal, Chetan Rajmohan, Saravan Lin, Qingwei Zhang, Chaoyun Zhang, Dongmei |
| author_facet | Yan, Haoran Chen, Yinfang Ma, Minghua Wen, Ming Lu, Shan Zhang, Shenglin Xu, Tianyin Wang, Rujia Bansal, Chetan Rajmohan, Saravan Lin, Qingwei Zhang, Chaoyun Zhang, Dongmei |
| contents | The ever-increasing demand for generative artificial intelligence (GenAI) has motivated cloud-based GenAI services such as Azure OpenAI Service and Amazon Bedrock. Like any large-scale cloud service, failures are inevitable in cloud-based GenAI services, resulting in user dissatisfaction and significant monetary losses. However, GenAI cloud services, featured by their massive parameter scales, hardware demands, and usage patterns, present unique challenges, including generated content quality issues and privacy concerns, compared to traditional cloud services. To understand the production reliability of GenAI cloud services, we analyzed production incidents from a leading GenAI cloud service provider spanning in the past four years. Our study (1) presents the general characteristics of GenAI cloud service incidents at different stages of the incident life cycle; (2) identifies the symptoms and impacts of these incidents on GenAI cloud service quality and availability; (3) uncovers why these incidents occurred and how they were resolved; (4) discusses open research challenges in terms of incident detection, triage, and mitigation, and sheds light on potential solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_08865 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Empirical Study of Production Incidents in Generative AI Cloud Services Yan, Haoran Chen, Yinfang Ma, Minghua Wen, Ming Lu, Shan Zhang, Shenglin Xu, Tianyin Wang, Rujia Bansal, Chetan Rajmohan, Saravan Lin, Qingwei Zhang, Chaoyun Zhang, Dongmei Distributed, Parallel, and Cluster Computing The ever-increasing demand for generative artificial intelligence (GenAI) has motivated cloud-based GenAI services such as Azure OpenAI Service and Amazon Bedrock. Like any large-scale cloud service, failures are inevitable in cloud-based GenAI services, resulting in user dissatisfaction and significant monetary losses. However, GenAI cloud services, featured by their massive parameter scales, hardware demands, and usage patterns, present unique challenges, including generated content quality issues and privacy concerns, compared to traditional cloud services. To understand the production reliability of GenAI cloud services, we analyzed production incidents from a leading GenAI cloud service provider spanning in the past four years. Our study (1) presents the general characteristics of GenAI cloud service incidents at different stages of the incident life cycle; (2) identifies the symptoms and impacts of these incidents on GenAI cloud service quality and availability; (3) uncovers why these incidents occurred and how they were resolved; (4) discusses open research challenges in terms of incident detection, triage, and mitigation, and sheds light on potential solutions. |
| title | An Empirical Study of Production Incidents in Generative AI Cloud Services |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2504.08865 |