Fluid Dynamic DNNs for Reliable and Adaptive Distributed Inference on Edge Devices
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916094568038400 |
|---|---|
| author | Xun, Lei Hu, Mingyu Zhao, Hengrui Singh, Amit Kumar Hare, Jonathon Merrett, Geoff V. |
| author_facet | Xun, Lei Hu, Mingyu Zhao, Hengrui Singh, Amit Kumar Hare, Jonathon Merrett, Geoff V. |
| contents | Distributed inference is a popular approach for efficient DNN inference at the edge. However, traditional Static and Dynamic DNNs are not distribution-friendly, causing system reliability and adaptability issues. In this paper, we introduce Fluid Dynamic DNNs (Fluid DyDNNs), tailored for distributed inference. Distinct from Static and Dynamic DNNs, Fluid DyDNNs utilize a novel nested incremental training algorithm to enable independent and combined operation of its sub-networks, enhancing system reliability and adaptability. Evaluation on embedded Arm CPUs with a DNN model and the MNIST dataset, shows that in scenarios of single device failure, Fluid DyDNNs ensure continued inference, whereas Static and Dynamic DNNs fail. When devices are fully operational, Fluid DyDNNs can operate in either a High-Accuracy mode and achieve comparable accuracy with Static DNNs, or in a High-Throughput mode and achieve 2.5x and 2x throughput compared with Static and Dynamic DNNs, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08943 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fluid Dynamic DNNs for Reliable and Adaptive Distributed Inference on Edge Devices Xun, Lei Hu, Mingyu Zhao, Hengrui Singh, Amit Kumar Hare, Jonathon Merrett, Geoff V. Computer Vision and Pattern Recognition Distributed inference is a popular approach for efficient DNN inference at the edge. However, traditional Static and Dynamic DNNs are not distribution-friendly, causing system reliability and adaptability issues. In this paper, we introduce Fluid Dynamic DNNs (Fluid DyDNNs), tailored for distributed inference. Distinct from Static and Dynamic DNNs, Fluid DyDNNs utilize a novel nested incremental training algorithm to enable independent and combined operation of its sub-networks, enhancing system reliability and adaptability. Evaluation on embedded Arm CPUs with a DNN model and the MNIST dataset, shows that in scenarios of single device failure, Fluid DyDNNs ensure continued inference, whereas Static and Dynamic DNNs fail. When devices are fully operational, Fluid DyDNNs can operate in either a High-Accuracy mode and achieve comparable accuracy with Static DNNs, or in a High-Throughput mode and achieve 2.5x and 2x throughput compared with Static and Dynamic DNNs, respectively. |
| title | Fluid Dynamic DNNs for Reliable and Adaptive Distributed Inference on Edge Devices |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.08943 |