Performance Characterization of Containerized DNN Training and Inference on Edge Accelerators
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911961276481536 |
|---|---|
| author | K., Prashanthi S. Hegde, Vinayaka Patchava, Keerthana Das, Ankita Simmhan, Yogesh |
| author_facet | K., Prashanthi S. Hegde, Vinayaka Patchava, Keerthana Das, Ankita Simmhan, Yogesh |
| contents | Edge devices have typically been used for DNN inferencing. The increase in the compute power of accelerated edges is leading to their use in DNN training also. As privacy becomes a concern on multi-tenant edge devices, Docker containers provide a lightweight virtualization mechanism to sandbox models. But their overheads for edge devices are not yet explored. In this work, we study the impact of containerized DNN inference and training workloads on an NVIDIA AGX Orin edge device and contrast it against bare metal execution on running time, CPU, GPU and memory utilization, and energy consumption. Our analysis shows that there are negligible containerization overheads for individually running DNN training and inference workloads. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_07220 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Performance Characterization of Containerized DNN Training and Inference on Edge Accelerators K., Prashanthi S. Hegde, Vinayaka Patchava, Keerthana Das, Ankita Simmhan, Yogesh Distributed, Parallel, and Cluster Computing Edge devices have typically been used for DNN inferencing. The increase in the compute power of accelerated edges is leading to their use in DNN training also. As privacy becomes a concern on multi-tenant edge devices, Docker containers provide a lightweight virtualization mechanism to sandbox models. But their overheads for edge devices are not yet explored. In this work, we study the impact of containerized DNN inference and training workloads on an NVIDIA AGX Orin edge device and contrast it against bare metal execution on running time, CPU, GPU and memory utilization, and energy consumption. Our analysis shows that there are negligible containerization overheads for individually running DNN training and inference workloads. |
| title | Performance Characterization of Containerized DNN Training and Inference on Edge Accelerators |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2312.07220 |