MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling

Documenti analoghi

• MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
  di: Zhang, Yu, et al.
  Pubblicazione: (2024)
• SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training
  di: Zhang, Jintao, et al.
  Pubblicazione: (2025)
• From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
  di: Pei, Zehua, et al.
  Pubblicazione: (2024)
• TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
  di: Lin, Weizhe, et al.
  Pubblicazione: (2025)
• Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
  di: Pei, Zehua, et al.
  Pubblicazione: (2025)