Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Peng, Jie, Cao, Zhang, Qu, Huaizhi, Zhang, Zhengyu, Guo, Chang, Zhang, Yanyong, Cao, Zhichao, Chen, Tianlong
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2410.14740