A Survey on Efficient Inference for Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914876808495104 |
|---|---|
| author | Zhou, Zixuan Ning, Xuefei Hong, Ke Fu, Tianyu Xu, Jiaming Li, Shiyao Lou, Yuming Wang, Luning Yuan, Zhihang Li, Xiuhong Yan, Shengen Dai, Guohao Zhang, Xiao-Ping Dong, Yuhan Wang, Yu |
| author_facet | Zhou, Zixuan Ning, Xuefei Hong, Ke Fu, Tianyu Xu, Jiaming Li, Shiyao Lou, Yuming Wang, Luning Yuan, Zhihang Li, Xiuhong Yan, Shengen Dai, Guohao Zhang, Xiao-Ping Dong, Yuhan Wang, Yu |
| contents | Large Language Models (LLMs) have attracted extensive attention due to their remarkable performance across various tasks. However, the substantial computational and memory requirements of LLM inference pose challenges for deployment in resource-constrained scenarios. Efforts within the field have been directed towards developing techniques aimed at enhancing the efficiency of LLM inference. This paper presents a comprehensive survey of the existing literature on efficient LLM inference. We start by analyzing the primary causes of the inefficient LLM inference, i.e., the large model size, the quadratic-complexity attention operation, and the auto-regressive decoding approach. Then, we introduce a comprehensive taxonomy that organizes the current literature into data-level, model-level, and system-level optimization. Moreover, the paper includes comparative experiments on representative methods within critical sub-fields to provide quantitative insights. Last but not least, we provide some knowledge summary and discuss future research directions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14294 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Survey on Efficient Inference for Large Language Models Zhou, Zixuan Ning, Xuefei Hong, Ke Fu, Tianyu Xu, Jiaming Li, Shiyao Lou, Yuming Wang, Luning Yuan, Zhihang Li, Xiuhong Yan, Shengen Dai, Guohao Zhang, Xiao-Ping Dong, Yuhan Wang, Yu Computation and Language Artificial Intelligence Large Language Models (LLMs) have attracted extensive attention due to their remarkable performance across various tasks. However, the substantial computational and memory requirements of LLM inference pose challenges for deployment in resource-constrained scenarios. Efforts within the field have been directed towards developing techniques aimed at enhancing the efficiency of LLM inference. This paper presents a comprehensive survey of the existing literature on efficient LLM inference. We start by analyzing the primary causes of the inefficient LLM inference, i.e., the large model size, the quadratic-complexity attention operation, and the auto-regressive decoding approach. Then, we introduce a comprehensive taxonomy that organizes the current literature into data-level, model-level, and system-level optimization. Moreover, the paper includes comparative experiments on representative methods within critical sub-fields to provide quantitative insights. Last but not least, we provide some knowledge summary and discuss future research directions. |
| title | A Survey on Efficient Inference for Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.14294 |