Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2112.15131 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911857658298368 |
|---|---|
| author | Lee, JunKyu Mukhanov, Lev Molahosseini, Amir Sabbagh Minhas, Umar Hua, Yang del Rincon, Jesus Martinez Dichev, Kiril Hong, Cheol-Ho Vandierendonck, Hans |
| author_facet | Lee, JunKyu Mukhanov, Lev Molahosseini, Amir Sabbagh Minhas, Umar Hua, Yang del Rincon, Jesus Martinez Dichev, Kiril Hong, Cheol-Ho Vandierendonck, Hans |
| contents | Deep learning is pervasive in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, face recognition, etc. However, deep neural networks demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of deep learning models, while the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient deep learning techniques in terms of model-, arithmetic-, and implementation-level techniques and identifies the research gaps for resource-efficient deep learning techniques across the three different level techniques. Our survey clarifies the influence from higher to lower-level techniques based on our resource-efficiency metric definition and discusses the future trend for resource-efficient deep learning research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2112_15131 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques Lee, JunKyu Mukhanov, Lev Molahosseini, Amir Sabbagh Minhas, Umar Hua, Yang del Rincon, Jesus Martinez Dichev, Kiril Hong, Cheol-Ho Vandierendonck, Hans Machine Learning Deep learning is pervasive in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, face recognition, etc. However, deep neural networks demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of deep learning models, while the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient deep learning techniques in terms of model-, arithmetic-, and implementation-level techniques and identifies the research gaps for resource-efficient deep learning techniques across the three different level techniques. Our survey clarifies the influence from higher to lower-level techniques based on our resource-efficiency metric definition and discusses the future trend for resource-efficient deep learning research. |
| title | Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2112.15131 |