Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models

Documenti analoghi

• PackMamba: Efficient Processing of Variable-Length Sequences in Mamba training
  di: Xu, Haoran, et al.
  Pubblicazione: (2024)
• TMA-Adaptive FP8 Grouped GEMM: Eliminating Padding Requirements in Low-Precision Training and Inference on Hopper
  di: Su, Zhongling, et al.
  Pubblicazione: (2025)
• H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
  di: Tang, Ding, et al.
  Pubblicazione: (2025)
• Towards Effective and Efficient Continual Pre-training of Large Language Models
  di: Chen, Jie, et al.
  Pubblicazione: (2024)
• Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
  di: Yang, Qinyu, et al.
  Pubblicazione: (2024)