Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908461278691328 |
|---|---|
| author | Miao, Xupeng Oliaro, Gabriele Zhang, Zhihao Cheng, Xinhao Jin, Hongyi Chen, Tianqi Jia, Zhihao |
| author_facet | Miao, Xupeng Oliaro, Gabriele Zhang, Zhihao Cheng, Xinhao Jin, Hongyi Chen, Tianqi Jia, Zhihao |
| contents | In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data. However, the computational intensity and memory consumption of deploying these models present substantial challenges in terms of serving efficiency, particularly in scenarios demanding low latency and high throughput. This survey addresses the imperative need for efficient LLM serving methodologies from a machine learning system (MLSys) research perspective, standing at the crux of advanced AI innovations and practical system optimizations. We provide in-depth analysis, covering a spectrum of solutions, ranging from cutting-edge algorithmic modifications to groundbreaking changes in system designs. The survey aims to provide a comprehensive understanding of the current state and future directions in efficient LLM serving, offering valuable insights for researchers and practitioners in overcoming the barriers of effective LLM deployment, thereby reshaping the future of AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_15234 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Miao, Xupeng Oliaro, Gabriele Zhang, Zhihao Cheng, Xinhao Jin, Hongyi Chen, Tianqi Jia, Zhihao Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data. However, the computational intensity and memory consumption of deploying these models present substantial challenges in terms of serving efficiency, particularly in scenarios demanding low latency and high throughput. This survey addresses the imperative need for efficient LLM serving methodologies from a machine learning system (MLSys) research perspective, standing at the crux of advanced AI innovations and practical system optimizations. We provide in-depth analysis, covering a spectrum of solutions, ranging from cutting-edge algorithmic modifications to groundbreaking changes in system designs. The survey aims to provide a comprehensive understanding of the current state and future directions in efficient LLM serving, offering valuable insights for researchers and practitioners in overcoming the barriers of effective LLM deployment, thereby reshaping the future of AI. |
| title | Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems |
| topic | Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2312.15234 |