LLM Inference Serving: Survey of Recent Advances and Opportunities
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914874908475392 |
|---|---|
| author | Li, Baolin Jiang, Yankai Gadepally, Vijay Tiwari, Devesh |
| author_facet | Li, Baolin Jiang, Yankai Gadepally, Vijay Tiwari, Devesh |
| contents | This survey offers a comprehensive overview of recent advancements in Large Language Model (LLM) serving systems, focusing on research since the year 2023. We specifically examine system-level enhancements that improve performance and efficiency without altering the core LLM decoding mechanisms. By selecting and reviewing high-quality papers from prestigious ML and system venues, we highlight key innovations and practical considerations for deploying and scaling LLMs in real-world production environments. This survey serves as a valuable resource for LLM practitioners seeking to stay abreast of the latest developments in this rapidly evolving field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12391 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLM Inference Serving: Survey of Recent Advances and Opportunities Li, Baolin Jiang, Yankai Gadepally, Vijay Tiwari, Devesh Distributed, Parallel, and Cluster Computing Artificial Intelligence This survey offers a comprehensive overview of recent advancements in Large Language Model (LLM) serving systems, focusing on research since the year 2023. We specifically examine system-level enhancements that improve performance and efficiency without altering the core LLM decoding mechanisms. By selecting and reviewing high-quality papers from prestigious ML and system venues, we highlight key innovations and practical considerations for deploying and scaling LLMs in real-world production environments. This survey serves as a valuable resource for LLM practitioners seeking to stay abreast of the latest developments in this rapidly evolving field. |
| title | LLM Inference Serving: Survey of Recent Advances and Opportunities |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence |
| url | https://arxiv.org/abs/2407.12391 |