The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws

Documenti analoghi

• Compression Scaling Laws:Unifying Sparsity and Quantization
  di: Frantar, Elias, et al.
  Pubblicazione: (2025)
• Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
  di: Bambhaniya, Abhimanyu Rajeshkumar, et al.
  Pubblicazione: (2024)
• SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learning
  di: Ohib, Riyasat, et al.
  Pubblicazione: (2024)
• The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse
  di: Sharma, Ekansh, et al.
  Pubblicazione: (2024)
• Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition
  di: Üyük, Cem, et al.
  Pubblicazione: (2024)