SymbolicLight V1: Spike-Gated Dual-Path Language Modeling with High Activation Sparsity and Sub-Billion-Scale Pre-Training Evidence

Documenti analoghi

• Training-Free Activation Sparsity in Large Language Models
  di: Liu, James, et al.
  Pubblicazione: (2024)
• MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes
  di: Zhao, Changsheng, et al.
  Pubblicazione: (2025)
• Exploring the Benefit of Activation Sparsity in Pre-training
  di: Zhang, Zhengyan, et al.
  Pubblicazione: (2024)
• Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
  di: Jiang, Minhao, et al.
  Pubblicazione: (2026)
• Spike No More: Stabilizing the Pre-training of Large Language Models
  di: Takase, Sho, et al.
  Pubblicazione: (2023)