Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yao, Jinghan, Jacobs, Sam Ade, Tanaka, Masahiro, Ruwase, Olatunji, Subramoni, Hari, Panda, Dhabaleswar K.
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2408.16978