Increasing Efficiency and Result Reliability of Continuous Benchmarking for FaaS Applications
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929462598172672 |
|---|---|
| author | Rese, Tim C. Japke, Nils Koch, Sebastian Pfandzelter, Tobias Bermbach, David |
| author_facet | Rese, Tim C. Japke, Nils Koch, Sebastian Pfandzelter, Tobias Bermbach, David |
| contents | In a continuous deployment setting, Function-as-a-Service (FaaS) applications frequently receive updated releases, each of which can cause a performance regression. While continuous benchmarking, i.e., comparing benchmark results of the updated and the previous version, can detect such regressions, performance variability of FaaS platforms necessitates thousands of function calls, thus, making continuous benchmarking time-intensive and expensive.
In this paper, we propose DuetFaaS, an approach which adapts duet benchmarking to FaaS applications. With DuetFaaS, we deploy two versions of FaaS function in a single cloud function instance and execute them in parallel to reduce the impact of platform variability. We evaluate our approach against state-of-the-art approaches, running on AWS Lambda.
Overall, DuetFaaS requires fewer invocations to accurately detect performance regressions than other state-of-the-art approaches. In 98.41% of evaluated cases, our approach provides equal or smaller confidence interval size. DuetFaaS achieves an interval size reduction in 59.06% of all evaluated sample sizes when compared to the competitive approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15610 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Increasing Efficiency and Result Reliability of Continuous Benchmarking for FaaS Applications Rese, Tim C. Japke, Nils Koch, Sebastian Pfandzelter, Tobias Bermbach, David Distributed, Parallel, and Cluster Computing In a continuous deployment setting, Function-as-a-Service (FaaS) applications frequently receive updated releases, each of which can cause a performance regression. While continuous benchmarking, i.e., comparing benchmark results of the updated and the previous version, can detect such regressions, performance variability of FaaS platforms necessitates thousands of function calls, thus, making continuous benchmarking time-intensive and expensive. In this paper, we propose DuetFaaS, an approach which adapts duet benchmarking to FaaS applications. With DuetFaaS, we deploy two versions of FaaS function in a single cloud function instance and execute them in parallel to reduce the impact of platform variability. We evaluate our approach against state-of-the-art approaches, running on AWS Lambda. Overall, DuetFaaS requires fewer invocations to accurately detect performance regressions than other state-of-the-art approaches. In 98.41% of evaluated cases, our approach provides equal or smaller confidence interval size. DuetFaaS achieves an interval size reduction in 59.06% of all evaluated sample sizes when compared to the competitive approaches. |
| title | Increasing Efficiency and Result Reliability of Continuous Benchmarking for FaaS Applications |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2405.15610 |