SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jia, Jinda, Xie, Cong, Lu, Hanlin, Wang, Daoce, Feng, Hao, Zhang, Chengming, Sun, Baixi, Lin, Haibin, Zhang, Zhi, Liu, Xin, Tao, Dingwen
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2410.15526