Quality at the Tail of Machine Learning Inference
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916136335966208 |
|---|---|
| author | Yang, Zhengxin Gao, Wanling Luo, Chunjie Wang, Lei Tang, Fei Wen, Xu Zhan, Jianfeng |
| author_facet | Yang, Zhengxin Gao, Wanling Luo, Chunjie Wang, Lei Tang, Fei Wen, Xu Zhan, Jianfeng |
| contents | Machine learning inference should be subject to stringent inference time constraints while ensuring high inference quality, especially in safety-critical (e.g., autonomous driving) and mission-critical (e.g., emotion recognition) contexts. Neglecting either aspect can lead to severe consequences, such as loss of life and property damage. Many studies lack a comprehensive consideration of these metrics, leading to incomplete or misleading evaluations. The study unveils a counterintuitive revelation: deep learning inference quality exhibits fluctuations due to inference time. To depict this phenomenon, the authors coin a new term, "tail quality," providing a more comprehensive evaluation, and overcoming conventional metric limitations. Moreover, the research proposes an initial evaluation framework to analyze factors affecting quality fluctuations, facilitating the prediction of the potential distribution of inference quality. The effectiveness of the evaluation framework is validated through experiments conducted on deep learning models for three different tasks across four systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_13925 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Quality at the Tail of Machine Learning Inference Yang, Zhengxin Gao, Wanling Luo, Chunjie Wang, Lei Tang, Fei Wen, Xu Zhan, Jianfeng Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Software Engineering Machine learning inference should be subject to stringent inference time constraints while ensuring high inference quality, especially in safety-critical (e.g., autonomous driving) and mission-critical (e.g., emotion recognition) contexts. Neglecting either aspect can lead to severe consequences, such as loss of life and property damage. Many studies lack a comprehensive consideration of these metrics, leading to incomplete or misleading evaluations. The study unveils a counterintuitive revelation: deep learning inference quality exhibits fluctuations due to inference time. To depict this phenomenon, the authors coin a new term, "tail quality," providing a more comprehensive evaluation, and overcoming conventional metric limitations. Moreover, the research proposes an initial evaluation framework to analyze factors affecting quality fluctuations, facilitating the prediction of the potential distribution of inference quality. The effectiveness of the evaluation framework is validated through experiments conducted on deep learning models for three different tasks across four systems. |
| title | Quality at the Tail of Machine Learning Inference |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Software Engineering |
| url | https://arxiv.org/abs/2212.13925 |