Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2401.02038 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911749295308800 |
|---|---|
| author | Liu, Yiheng He, Hao Han, Tianle Zhang, Xu Liu, Mengyuan Tian, Jiaming Zhang, Yutong Wang, Jiaqi Gao, Xiaohui Zhong, Tianyang Pan, Yi Xu, Shaochen Wu, Zihao Liu, Zhengliang Zhang, Xin Zhang, Shu Hu, Xintao Zhang, Tuo Qiang, Ning Liu, Tianming Ge, Bao |
| author_facet | Liu, Yiheng He, Hao Han, Tianle Zhang, Xu Liu, Mengyuan Tian, Jiaming Zhang, Yutong Wang, Jiaqi Gao, Xiaohui Zhong, Tianyang Pan, Yi Xu, Shaochen Wu, Zihao Liu, Zhengliang Zhang, Xin Zhang, Shu Hu, Xintao Zhang, Tuo Qiang, Ning Liu, Tianming Ge, Bao |
| contents | The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training and deployment within this context. Low-cost training and deployment of LLMs represent the future development trend. This paper reviews the evolution of large language model training techniques and inference deployment technologies aligned with this emerging trend. The discussion on training includes various aspects, including data preprocessing, training architecture, pre-training tasks, parallel training, and relevant content related to model fine-tuning. On the inference side, the paper covers topics such as model compression, parallel computation, memory scheduling, and structural optimization. It also explores LLMs' utilization and provides insights into their future development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_02038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding LLMs: A Comprehensive Overview from Training to Inference Liu, Yiheng He, Hao Han, Tianle Zhang, Xu Liu, Mengyuan Tian, Jiaming Zhang, Yutong Wang, Jiaqi Gao, Xiaohui Zhong, Tianyang Pan, Yi Xu, Shaochen Wu, Zihao Liu, Zhengliang Zhang, Xin Zhang, Shu Hu, Xintao Zhang, Tuo Qiang, Ning Liu, Tianming Ge, Bao Computation and Language The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training and deployment within this context. Low-cost training and deployment of LLMs represent the future development trend. This paper reviews the evolution of large language model training techniques and inference deployment technologies aligned with this emerging trend. The discussion on training includes various aspects, including data preprocessing, training architecture, pre-training tasks, parallel training, and relevant content related to model fine-tuning. On the inference side, the paper covers topics such as model compression, parallel computation, memory scheduling, and structural optimization. It also explores LLMs' utilization and provides insights into their future development. |
| title | Understanding LLMs: A Comprehensive Overview from Training to Inference |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.02038 |