Towards Fast Setup and High Throughput of GPU Serverless Computing
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929324430458880 |
|---|---|
| author | Zhao, Han Cui, Weihao Chen, Quan Zhang, Shulai Li, Zijun Leng, Jingwen Li, Chao Zeng, Deze Guo, Minyi |
| author_facet | Zhao, Han Cui, Weihao Chen, Quan Zhang, Shulai Li, Zijun Leng, Jingwen Li, Chao Zeng, Deze Guo, Minyi |
| contents | Integrating GPUs into serverless computing platforms is crucial for improving efficiency. However, existing solutions for GPU-enabled serverless computing platforms face two significant problems due to coarse-grained GPU management: long setup time and low function throughput.
To address these issues, we propose SAGE, a GPU serverless framework with fast setup and high throughput. First, based on the data knowability of GPU function ahead of actual execution, SAGE first devises the parallelized function setup mechanism, which parallelizes the data preparation and context creation. In this way, SAGE achieves fast setup of GPU function invocations.Second, SAGE further proposes the sharing-based memory management mechanism, which shares the read-only memory and context memory across multiple invocations of the same function. The memory sharing mechanism avoids repeated data preparation and then unnecessary data-loading contention. As a consequence, the function throughput could be improved. Our experimental results show that SAGE reduces function duration by 11.3X and improves function density by 1.22X compared to the state-of-the-art serverless platform. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14691 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Fast Setup and High Throughput of GPU Serverless Computing Zhao, Han Cui, Weihao Chen, Quan Zhang, Shulai Li, Zijun Leng, Jingwen Li, Chao Zeng, Deze Guo, Minyi Distributed, Parallel, and Cluster Computing Integrating GPUs into serverless computing platforms is crucial for improving efficiency. However, existing solutions for GPU-enabled serverless computing platforms face two significant problems due to coarse-grained GPU management: long setup time and low function throughput. To address these issues, we propose SAGE, a GPU serverless framework with fast setup and high throughput. First, based on the data knowability of GPU function ahead of actual execution, SAGE first devises the parallelized function setup mechanism, which parallelizes the data preparation and context creation. In this way, SAGE achieves fast setup of GPU function invocations.Second, SAGE further proposes the sharing-based memory management mechanism, which shares the read-only memory and context memory across multiple invocations of the same function. The memory sharing mechanism avoids repeated data preparation and then unnecessary data-loading contention. As a consequence, the function throughput could be improved. Our experimental results show that SAGE reduces function duration by 11.3X and improves function density by 1.22X compared to the state-of-the-art serverless platform. |
| title | Towards Fast Setup and High Throughput of GPU Serverless Computing |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2404.14691 |