Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.10847 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929541460525056 |
|---|---|
| author | Gong, Yifan Wu, Yushu Zhan, Zheng Zhao, Pu Liu, Liangkai Wu, Chao Tang, Xulong Wang, Yanzhi |
| author_facet | Gong, Yifan Wu, Yushu Zhan, Zheng Zhao, Pu Liu, Liangkai Wu, Chao Tang, Xulong Wang, Yanzhi |
| contents | Two-stage object detectors exhibit high accuracy and precise localization, especially for identifying small objects that are favorable for various edge applications. However, the high computation costs associated with two-stage detection methods cause more severe thermal issues on edge devices, incurring dynamic runtime frequency change and thus large inference latency variations. Furthermore, the dynamic number of proposals in different frames leads to various computations over time, resulting in further latency variations. The significant latency variations of detectors on edge devices can harm user experience and waste hardware resources. To avoid thermal throttling and provide stable inference speed, we propose Lotus, a novel framework that is tailored for two-stage detectors to dynamically scale CPU and GPU frequencies jointly in an online manner based on deep reinforcement learning (DRL). To demonstrate the effectiveness of Lotus, we implement it on NVIDIA Jetson Orin Nano and Mi 11 Lite mobile platforms. The results indicate that Lotus can consistently and significantly reduce latency variation, achieve faster inference, and maintain lower CPU and GPU temperatures under various settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_10847 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lotus: learning-based online thermal and latency variation management for two-stage detectors on edge devices Gong, Yifan Wu, Yushu Zhan, Zheng Zhao, Pu Liu, Liangkai Wu, Chao Tang, Xulong Wang, Yanzhi Computer Vision and Pattern Recognition Machine Learning Two-stage object detectors exhibit high accuracy and precise localization, especially for identifying small objects that are favorable for various edge applications. However, the high computation costs associated with two-stage detection methods cause more severe thermal issues on edge devices, incurring dynamic runtime frequency change and thus large inference latency variations. Furthermore, the dynamic number of proposals in different frames leads to various computations over time, resulting in further latency variations. The significant latency variations of detectors on edge devices can harm user experience and waste hardware resources. To avoid thermal throttling and provide stable inference speed, we propose Lotus, a novel framework that is tailored for two-stage detectors to dynamically scale CPU and GPU frequencies jointly in an online manner based on deep reinforcement learning (DRL). To demonstrate the effectiveness of Lotus, we implement it on NVIDIA Jetson Orin Nano and Mi 11 Lite mobile platforms. The results indicate that Lotus can consistently and significantly reduce latency variation, achieve faster inference, and maintain lower CPU and GPU temperatures under various settings. |
| title | Lotus: learning-based online thermal and latency variation management for two-stage detectors on edge devices |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2410.10847 |